PanoBot

At least I got front page. Anyway, if you like my project, I'd really appreciate it if you could head to my Printables page, like the design, and download the print files. Enough downloads and likes will get me more filament for my next project.

This didn't quite turn out how I had hoped. The robot chasis turned out far better than I expected, but the video leaves too much to be desired. I like the VR view, but the resolution is just plain crap and barely useable. It looks far worse in person than it does in the videos, due to Quest 2's screen recording weirdness. My plan is to keep the chasis, but redo the camera setup. I'm going to ditch the panorama lens. Back when lens-based panorama videos came out, expensive lenses were still cheaper than having more than one camera, and easier to manage. That's not true anymore. Since each esp32cam is only $8.00, and they can only stream at resolutions too low for good panoramas, I'm going to switch to multiple cameras in a circle.

I'll measure the view angle of the cameras and buy enough to get a full 360. I'll have one set up as the webserver and command and control for the others. I may have each camera have its own websocket connection to the webpage.

As to the VR code, that'll stay largely the same, but I can lose the shader. I'll just have the panorama view be multiple cylinder segments (easy to do in A-Frame.js) and crop the textures in-memory using offsets. It'll be way easier.

As to the panorama lens, it won't go to waste. There's a neat navigation algorithm that uses single-pixel height panoramas and it will very likely fit in the RAM of an ESP32. And I've been itching to give my roomba a lobotomy.

I'm calling this done for now. I'm not satisfied with the stream quality. I made a few final tweaks to increase the resolution without sacrificing framerate. As promised, here's some demos of using it in Oculus Quest. But before we get to that, more complaints! Screen recording in Quest is janky as hell! I couldn't record more than two minutes or it crashes and losing the video I recorded! So after a few attempts, here's what I could manage...

This video demonstrates calibrating the panorama stream in VR. Note that the panorama gets correctly calibrated, and in my headset it looks correct, however the screen recorder captures it weird. It's doing something odd to the WebGL textures that shows up in the recording, but not in the actual headset view. But it is what it is.

In the second video, it just continues where the previous left off.. I realized I didn't give a very good view of the virtual vehicle, so this one opens with that.

I've gotten the old control scheme to coexist with the new Oculus touch controls and I've added the 2d touch control mode. The last thing I have to do is figure out how to force a high resolution texture on my shader in VR mode. Right now it shows the panorama at an unacceptably low resolution. And A-Frame's documentation on shader textures is dismal. But the 2D mode isn't using my shader, it's using plain old inefficient JavaScript and canvas, so it's full resolution. Upload of latest version later today. Hopeful I'll have the resolution problem fixed by then. Stay tuned.

Debugging VR for Occulus Quest is a bit annoying. Anyway, I got video streaming working in Quest VR and the driving controls are now working. Now I just need to figure out a convenient way to adjust the panorama dimensions with the Quest touch controllers. Also, I finished the 2D touch controls. I'll upload the updated code for that tomorrow. For now, enjoy this new demo of 2D mode, as I (poorly) drift around Flat Eric.

As this is an ongoing project, I'm getting ready to submit for the prize as is. In its current state, it'll work with a google cardboard viewer and xbox controller. I will be updating the code over the next few days to improve the web frontend. Speciffically, I'll be adding a mobile 2D touch control mode, as well as add support for Quest 2. It does not work in the Quest 2 web browser in VR mode. For some reason the main thread pauses when it enters VR, so the textures never update with the video feed and the controls do nothing as I haven't figured out the Quest controller mappings. Stay tuned...