Immersive VR on Rpi to control a Hero with sensors

DOF (Degree Of Freedom) Sensors

When I started this project, I hesitated a lot on what DOF sensors I should buy. It was obvious to me that I would need a 9 degrees of freedom to avoid the infinite rotating effect that 6 DOF would create.

                                          

But then I discovered this code with a demo running on Free_IMU. It was perfectly still! Even after many random moves, the Yaw axe seems to be only a few degrees different while the Pitch and Roll stays perfectly level (see the plane picture above if you don't understand the three axes) which is perfectly fine for me for now... In addition, Eulers angles can be calculated with a "home reference position". Very useful for calibrating our sensors!

Hardware

The schematic of the device is above. Here are the components to build one :

• AirBoard, Arduino FIO or DIY solution. FIO Arduino bootloader runs with an 8MHz crystal (instead of 16MHz on the UNO) which allows it to be used in lower voltage. So if you have an Arduino project running on a battery, it is a great strategy to directly start with a 8MHz board (Or you can use the internal 8Mhz oscillator). Of course if your project needs the full performance of the UNO, this may create problems, but most of them don't...
• MPU 6050, 6DOF sensor which communicates with the Arduino over I2C
• NRF24L01+ Bluetooth chip which communicates with SPI. This board became very famous because of its great manufacturer NORDIC (which is a reference in BT devices) but also because of its amazingly cheap price : less than 1$! If we also consider the fact that it allows multi-channel communication, you have yourself a perfect toy to play with!
• Dip switch to select devices
• Lipo Battery
• Headers and wires to charge the battery and push the program.

For debugging this project, I used several Arduino Airboard from a Kickstarter project. This amazing little board embeds a FIO, a Battery, a power switch, LEDs... Plus, it accepts remote programming over a dedicated BLE chip and a USB dongle. I recommend using those for debugging embedded battery projects, it will save you a lot of time!

Software

Processing test software (Also in Github here) I already talked a bit about the library for the 6DOF sensor. If you want to give it a try, download this code and wire the sensor to an Arduino board. Once processing, start the IMU interface and you can start playing. The "h" key will recalibrate the sensor.

The Euler angles problems & solutions

Playing with 3D object rotations is a hard work. Using Arduino libraries and Pi3D helped me a lot to control the movement for one object. But I am still debugging a bit when the movement of one sensor depend on the movement of another. For example, hands movements depend on arms movements. So the angles of the hands have to be "HANDS_ANGLES - ARMS_ANGLES = REAL_HANDS_ANGLES". As Euler values aren't linear, "-ARMS_ANGLES" isn't the opposite of "ARMS_ANGLES"...

To implement this part, I worked on two solutions :

• SOLUTION 1 : The nodes send only the 4 quaternions on Bluetooth. The central receive and calculate Euler angles. When the quaternions of the hands is received, the Euler is calculated depending on the arms quaternions as "home reference". This solution works, but the central needs more time to calculate.
• SOLUTION 2 : My first choice was to calculate Euler angles inside the nodes and send them over Bluetooth (which is nice because I still have one empty byte on the buffer for the button). In Pi3D I tried to implement the Hands depending on the arm, but it's still not working perfectly right now. If you are interested in this topic, you can take a look on the conversation I had with paddyg here.

Calibration

As you can see, those 6-DOF sensors are quite precise. But without a compass, they need a YAW calibration. On Free_IMU, the key 'h' set a "home calibration". In my solution, one of the Nunchuck button trigger the "home calibration" in the central. So if your sensors seems to have a wrong orientation, just press the button and take the same position as your avatar when all the angles are "0,0,0". To simplify, I always draw my avatars in the same initial position.

Bluetooth library

There are a lot of NRF24 libraries online. Some didn't work with my application, and more specifically because of the 6DOF library delay which probably caused synchronization problems... So use this one, it seems to work very well with other libraries ;)

DIP switch

Using multi-transceiver Bluetooth, you are going to need different addresses on each node. That's why the DIP switch is here. You can load the same program in every 6DOF node and only play with the switches to give them different addresses. You forgot to charge one of your nodes? No problem, take another one, set the same DIP address, and that's it!

Packaging

Sorry, not finished this part yet, I will send some STL 3d print files as soon as I can

Purpose of the device :

1. Records the body's movement through the 6DOF sensors over I2C
2. Analyses the data and creates the 4 Quaternions to determine the position
3. Calculates the three Euler angles. In case a reference point has been defined, the Euler angles will also depend on it
4. Casts each Euler angle floats into a byte, so we can send all three in one message (4 bytes buffer limit). This action reduces the number of possible values to 256 per angle... but it speeds up the communication a lot.
5. Sends the Buffer
6. Enters shortly into Reception Mode, so if a new referencing point has been set, it will be taken into consideration. (This part is changing on the last version)

Have you ever used HID devices?

Of course! You are probably using several right now!

HID stands for "Human Interaction Devices", it's the USB protocol used for Keyboards and Mouses and it's probably the easiest way to interact with a computer as it's recognized by any OS system.

Some Arduino boards like Leonardo or Micro aren't using the original ATmega328p AVR chip as UNO. They implement a Atmega32u4. The name doesn't seems very different... but this controller will give you the possibility to communicate with the USB port of a computer through HID. When you play with an Arduino UNO, you need to add a chip to interconnect the USB (like FTDI or 32u2) as HID is incompatible (except V-USB for AVR library, but the implementation is slower).

                                                

This HID capability will provide us with a great way to interface between Bluetooth and Raspberry Pi because we can re-use some of the Mouse or Keyboard libraries for an easy interaction and also use a Serial terminal communication for more complex data transfers.

Hardware

So, the hardware is pretty simple and can be re-used in many different projects:

• A Leonardo or Micro Arduino board. I used Micro to save space, but the Leonardo has a 3.3V power supply, which is useful for powering the Bluetooth. If you use a Micro, I recommend you to add an external 3.3V VCC. 
• A NRF24L01+ on the SPI port (Pin 3/10/11/12/13, follow the schematic under). 
• A button. To avoid headache, this safety button can block the HID. Sometimes a wrong loop can make your board send thousands of keyboard and mouse commands!!! Its only function is to say "send nothing if the button is pressed!" 
• I used a tiny breadboard, some male/female headers and some wires to plug in everything together (See the picture above). I will try to optimize it and make a 3D printed box later when I will have finished other more important things :p

                                         

Software

All the software is available on this Github. For the Bluetooth HID, I had to manually add the libraries mouse/keyboard/Joystick (I use Arduino 1.6.9). This doesn't really make sense, as they are supposed to be already implemented...

                                        

You can try removing the first three includes, then select Arduino Micro or Leonardo board and compile (Be careful, it's not going to work with UNO!). If it says keyboard, mouse or joystick not recognised, you have to download the library yourself "write arduino keyboard.h in google". Then you move them inside the Arduino installation folder (java/library in mac) and restart Arduino software!

The libraries you need depend on what you want. At first I was using a joystick lib, but it seemed to slow the Python code down. So I made another function to send data through Serial as well.

The mouse HID is very useful if you want to follow the movement of the head. So we keep the possibility of controlling our Python program with a normal mouse or with the Micro device.