I'm calling this done for now. I'm not satisfied with the stream quality. I made a few final tweaks to increase the resolution without sacrificing framerate. As promised, here's some demos of using it in Oculus Quest. But before we get to that, more complaints! Screen recording in Quest is janky as hell! I couldn't record more than two minutes or it crashes and losing the video I recorded! So after a few attempts, here's what I could manage...

This video demonstrates calibrating the panorama stream in VR. Note that the panorama gets correctly calibrated, and in my headset it looks correct, however the screen recorder captures it weird. It's doing something odd to the WebGL textures that shows up in the recording, but not in the actual headset view. But it is what it is.

In the second video, it just continues where the previous left off.. I realized I didn't give a very good view of the virtual vehicle, so this one opens with that.