VR/AR interaction devices

Most of the parts seem to fit together. I made a silly mistake with one of the dimensions as I do not use the arduino in the CAD and I did not notice one of the assembly issues .

Overall it is pretty compact and I think this is ok for a first prototype. Improving the form factor could be a good project to build upon. This will be atleast partially a job for someone interested in electronics as we might want to move away from using an arduino and breadboard to something more compact.

You will need a drill and drill set for the assembly. Most of the features used to align and fix the pieces are cylindrical in nature to allow you to make the small adjustments with a drill( as opposed to a mill)-this is required when the holes don't print perfectly.

I am also going to have to re-design the IMU mount as I have a new idea to mount it which can result in a smaller device.

I am attaching some pictures which will help explain the assembly ( full instructions will be available only after I reach a somewhat final design. I am posting these updates because some of you may want to suggest improvements on the design). My hands are shown for reference. (I have small hands).

For the modular design of the wand, it will be more than just swapping out buttons or components. I am thinking along the lines of attaching additional frames. So for example, you can take the wand and take a frame for a two handed submachine gun and attach the wand as one of the handles for the gun.(We will now use the orientation data from the wand which is more reliable).

Then because our exoskeleton glove can haptically render squishy features, it can be used to create a virtual trigger(haptic feedback) -(not possible with braking mechanism).

So why do we need to attach that frame? Because when you hold a physical gun, it places constraints on your hands. I.e. Both hands need to be at a fixed distance from each other in physical space. If we do not have a frame to grip with both arms then there will be a mismatch between what you feel and what you see. This is not only bad for immersion but creates real problems for input. Since the gun frame is supposed to be rigid, both palms end up with a fixed orientation with respect to each other. If we do not have the frame, the arms can move with respect to each other. Now how do we take the input? The orientation of only one? That is a one handed gun. It will cause a mismatch. Do we use relative position to calculate the angle? That is somewhat better but position tracking is unreliable. Furthermore, even with position tracking, it is still un-natural. Therefore we need the gun frame.

So why not just create a gun VR device? That might be a good project for someone to work on. I am trying to keep it modular because I want people to be able to take this one device and do lots of things with it. VR is already expensive and it will be nice if we can re-use components as much as possible.

Explanation of developments in wand design and setup of IMU

I am first trying out the BNO055 IMU. I have managed to get some sensor readings( one of my arduinos broke). I was hoping to have good orientaton readings even if the position is going to drift. I note that the damping is not amazing, so if I disturb my IMU and then leave it alone, it takes a while for the orientation to settle down.

Nonetheless I am pleased with the data so far because I believe that the VR application can handle all of these issues. Example- in a shooting game you will you some kind of autoaim and therefore even if the reading is not perfect we are just trying to make sure we can get reasonable data to understand user intent.

Also, just a note to dummies like me who are following the guide online- use the PDF for the IMU instructions, the wiring is somewhat different if you dont use the same arduino as the one on the webpage(uno).

On the wand design front, I am posting some pictures which will explain the design. I can already see that the breadboard holder and battery holder can all be integrated into the frame. This will allow me to shave some mm to get more space/smaller device. (Do note that I am designing the CAD files so that you get pretty stiff parts which will not warp and will still fit after cooling shrinkage and therefore many of the thicknesses may seem too large. We can certainly use different materials and manufacturing processes for a more compact build but hopefully this is something which people can make with the tools they have at home)

Update: I am still working on that IMU setup, everything does not seem to be calibrated perfectly.

Initial design concept for the glove:

For a glove exoskeleton, ideally we would like to be able to separate the finger into two links and give actuation for both links. However since I need to make a wearable, using the current set of commonly and cheaply available actuators, I will only be providing a single link to apply the force on the index finger.

Since I want to be able to apply a variable impedance at various points in the coordinate space for the finger. I am currently working along the lines of a motor and spring combination.

The reason why I am using the spring is because of time response and ease of control. One of the main use-cases in VR is currently gaming. Gaming interactions are very quick. This means we need to be able to apply forces on the hand very quickly. If I use only a servo and combine it with a force or pressure sensor at the finger tip(or something along those lines) then there will be a time lag between when the force or movement is detected by the sensor, this information is sent to Micro-controller, then either processed by VR device then sent back to controller and then actuated by the motor.

However with the spring device, when the finger is trying to move, the physics of the spring will automatically apply a force on the finger. I will sense this change in force and then control the motor to change the deformation of the torsional spring. This will ensure a responsive actuator with fast feedback and at the same time allow for generation of impedance in a more natural way.

I am thinking of using a rotation sensor to sense the deformation in the spring. It looks like some mechanical encoders are available cheaply however these encoders have only 24 pulses per revolution. I think this is a somewhat low resolution. I am thinking of either using a gear between the shaft and encoder to give me a higher effective resolution OR to use an optical encoder.

Do let me know if you have any suggestions on cheap,compact rotation sensor with atleast 4 degrees resolution.

Checked out the 3D printer available to me. Noticed a lot more shrinkage than I am used to. I have made available the parts that work after accounting for shrinkage.

The part is a holder for the breadboard. This part will attach onto the wand handle part. I am not sticking the breadboard directly onto the wand handle because I do not want it to come lose when the wand is moved around. I also want to be able to re-use all components on different projects. I like to avoid getting double sided tape stuck to the bottom as many times a little residue builds up when you remove the tape, do this a couple of times and you change the effective thickness of your breadboard.

I believe that my system will be able to or should attempt to enable the following types of interactions:

Gaming:

1) Using a bow and arrow with some kind of haptic cues

2) Using a sword and shield or a knife( This is a classic case of where a full exoskeleton system is more immersive but more on this later on)

3) Using different types of guns- pistols,shotguns, shoulder fired rocket, submachine guns etc.

4) Boxing?

Non gaming:

1) Pointing, pressing/touching, grasping

2) Button oriented input

Locomotion problem:

Hopefully I will be able to enable a system which works better than teleportation and overcomes some issues with treadmills.

Edit: Note that simulator sickness is supposed to be caused by mismatch visual, vestibular and proprioceptive senses. The treadmill allows you to trick your proprioceptive sense but your vestibular sense is less affected. My proposed method of locomotion also allows you to stimulate your proprioceptive sense(to a smaller extent). I intend to test whether this is sufficient to deal with the simulator sickness issue.

I am going to try to keep some parts of the wand device modular so that people can customize it to use only the parts that they need for their specific implementation.