First, please enjoy the Holokart demo video so you can see this in action! It's a real life version of Mario Kart, where you get into a physical (as opposed to virtual) go-kart, and put on a Microsoft Hololens augmented reality visor. As you race around the track, you'll see holographic power-ups that you can collect by driving through them... and thanks to some electronics I've added to the go-karts, those virtual power-ups can actually boost the speed of your real-life go-kart, or slow the speed of your competition (depending on which power-up you get).

Since the hackaday crowd is an intellectually curious group, here are some behind-the-scenes details on the technology here.

One thing I've learned the Microsoft Hololens does insanely well is inside-out SLAM (simultaneous localization and mapping). If you're not familiar, this is a concept from autonomous robots where there are a variety of sensors and as the device moves around it creates a map of its surroundings and keeps track of where it is in those surroundings. The Hololens uses visible light cameras, an interial measurement unit, and also a structured grid of infrared light (along with an IR camera) and combines the data coming back from all of those sensors to produce an astoundingly accurate map of the environment despite the possibility of some error in one of those sensor streams. In order to maximize the Hololens' ability to track its location at "high speed" (10-15 miles per hour), we placed posterboards around the track with black rectangular stickers on them. The sharp corners of those stickers are the visual "features" that visual SLAM algorithms use to track movement. In the final production system, those posterboards will be replaced by cardboard standups of cars, racers, audience members, checkered flags... things that fit the racing theme and have a purpose (track decoration) beyond just providing sharp features for the Hololens to track. Those cardboard cutouts would be 3 as dimensional as possible, so that the grid of IR light and IR camera has good three-dimensional features to detect as well. I am confident that those improvements, along with switching to the Hololens 2 and improving lighting conditions at the track, will significantly reduce or eliminate the "lag" that you see in the video, where the virtual characters lag behind the racers by a few frames.

The coolest part of this project, in my opinion, is the interaction between the real-life go-karts and the virtual Hololens game.  For example, if you use a speed-boosting power-up (the catchup and mustard "jetpack" you see in the demo video), your go-kart will speed up for a set amount of time. Likewise, if you drop an obstacle (the bowling pins you see in the demo) and an opponent runs into it, their go-kart will stop for a moment, allowing you to get ahead of them in the race. This is accomplished through remote control. Modern electric go-karts have remote controls that the venues use for example to stop all racers in the event of an accident, or to slow down or speed up karts depending on the ability of the drivers. The go-kart system that I implemented this with uses a 433 Mhz OOK system that was straightforward to decode. I implemented a 433 Mhz transceiver that was attached to the game server using USB serial, that allows the game server to selectively slow down or speed up karts in order to make the power-ups actually control the speed of the karts. 

The remaining issue was how can the user activate the power-up? I added an "activate power-up" button to the go-karts. This button is connected to an arduino with a wifi shield, so it can tell the game server when a user has pressed the "activate" button. Each arduino was programmed so that it knows which kart it's in, and the hololens units know which kart they are associated with because of a "game start" procedure that involves pointing the Hololens at an identifying QR code on the back of the kart's electronics...