Captain's Log: Stardate 77464.6

I know a few things about The Bug now.

First, I can reproduce it a bit more reliably. What I found is that it appears ~19 seconds after startup, and then every 19 seconds after that, for approximately 1-2 seconds at a time. But there are two conditions: (1) it only happens if the plugin is making sound, and (2) on some launches, it never happens. This is really weird; something is happening at startup that randomly enables the bug, and if that doesn't happen, the bug will never appear during that session. But if the bug is triggered, it will always happen every 19 seconds.

I've been doing experiments to narrow down the source of the bug. I currently know that it appears every 19 seconds even if I elide kernel execution for 10 seconds in the middle. I tried incrementing the sample count 2 samples per 1 actual sample, and it still happened every 19 seconds. I looked at everywhere state is accumulated, and ruled it out.

The weird noise it makes is affected by microphone gain. This means that the problem must be in the GPU code (or possibly the CPU setup work for the GPU code). But then I ran an experiment where I ripped out all the physics code and just made each mass vibrate on a pure sin() wave, and the weird noise STILL occurred. This leads me to think that the problem is with the synchronization for the work unit's local memory. But this is not particularly consistent with the "every 19 seconds, on some launches" characteristic.

The last thing I did today was to keep ripping out GPU code. I think I may have finally found a state where I've ripped out enough that The Bug no longer occurs. But I haven't had time yet to start adding code back in slowly to see when it reappears. This is pretty promising, though.