dysfun@treehouse.systems ("gaytabase") wrote:
for example, i use a slightly racy queue, because it's so much faster on average. the downside is i have to detect the race and handle it. and i calculated that it's going to happen about 0.1-0.2% of the time. sure, that is absolutely a tiny fraction of the time and you'll probably make it back on all the other executions, but on the other hand, a small percentage are destined to wait a really long time (up to milliseconds!). and we just spinloop through this usually!





![[codspeed benchmarking comment saying that one PR that rewrites some of the core logic in rust will increase performance across our benchmarks by 37%. each of the tests are a processing graph run in different ways - a synchronous runner just calls the graph... synchronously.. in order, and a ZMQ runner is a multiprocess runner that runs each node concurrently, swapping events over zeromq. i figure the details are probably not important to anyone but me, but if you are curious about the actual numbers in the table, HTML is probably easier for screenreaders and tables than i can render here, so that's over here: https://github.com/miniscope/noob/pull/243#issuecomment-4977387758 ]](https://files.mastodon.social/cache/media_attachments/files/116/922/572/322/326/782/original/ebd1794cb8d3d583.png)
![[benchmarks that i ran locally on just the part that was swapped out - regular graphs are scheduled 4-5x faster than before, and large graphs are more like 10x]](https://files.mastodon.social/cache/media_attachments/files/116/922/572/396/894/397/original/58348c8898a8fdd1.png)



