Ergonomic handheld mouse / keyboard alternative

I posted a video showing the basics of how to use it as a simple wireless keyboard, including pairing it in Windows 10 and testing the pressed keys:

I was going to order parts to build a 2nd device, so I can have a keyboard in my left hand (Spacebar, Enter, Escape and Backspace), and a mouse in my right hand (Left-click, Right-click, Wheel scroll up, Wheel scroll down).

But it looks like the DRV5053VAQDBZR magnetic sensor that I used is currently out of stock at Digikey, Mouse, Farnell / Element14 and various other distributors! It looks like they won't be back in stock for a few months. Luckily the manufacturer Texas Instruments in USA still has several thousand left in stock, and TI is willing to sell them in packs of 5 sensors for $4.80 with free shipping to various countries.

So if you are having trouble finding these specific magnetic sensors, try

https://store.ti.com/DRV5053VAQDBZRQ1.aspx

If TI stops producing this sensor one day, there are a few other magnetic sensors with similar sensitivity that should work OK as a replacement.

After posting this project on the web I've had a few questions about other use cases for this device:

Can it be used as a VR controller?

Not really because it only has a 6-DOF IMU, and for a VR controller you would want either a 9-DOF IMU (with a quaternion-based Kalman filter or similar) to sense the position or a different position detection sensor. If you added this to the board yourself then I think it actually would be a good VR input device, since it gives you multiple keys and/or analog inputs.

Can it be used as a basic chording keyboard?

Yes, it would be very easy to use it as a 4-key chording input. Just modify the lines at the bottom of main.cpp to processing multiple variables such as:

if ( (fing1 > FINGER1_PRESS && !press_keyup[1])
  && (fing2 > FINGER2_PRESS && !press_keyup[2]) ) {

I personally haven't used chording keyboards so I haven't implemented it, but there's no reason for it not to work.

Can it be used to control a quadcopter drone?

Yes potentially, either by using the analog finger positions or the analog motion inputs. But since the code is configured to send BLE keyboard or mouse inputs, you would need to do some work for it function as a drone controller, unless if you find a drone that already takes BLE keyboard or mouse or joystick inputs (I'm not sure if that exists?). Using it as HID Joystick to control a drone flight simulator on a PC would be a relatively easy option.

The hardware in Tiny BLE is capable of being a BLE keyboard and mouse, but unfortunately they don't provide source code for this right out of the box. There is a demo project for Tiny BLE using the mbed online compiler (https://github.com/Seeed-Studio/Tiny_BLE), and I also found sample code of a BLE keyboard and of a BLE mouse using mbed (https://github.com/jpbrucker/BLE_HID), but they aren't designed specifically for Tiny BLE. So they need to be merged together.

Unfortunately I didn't find good sample code that allows a single Bluetooth BLE device to behave as both a keyboard and mouse at the same time. The only code I did find that is capable of both at the same time was some code on a forum at https://community.cypress.com/thread/11130 that uses theCypress PSoC Creator graphical tool, and therefore the generated code is quite large and tricky to understand.

So my device will act as either a keyboard or a mouse, but not both at the same time.

Anyway, after a bunch of trial & error (is it just me or is Bluetooth extremely unreliable?!), I got it to work as a keyboard, and then with a code rebuild, as a mouse! I used the free mbed OS and online compiler, because BLE apparently doesn't easily fit into the restricted flash size that free versions of compilers such as Keil and IAR support.

While building this project, I accidentally dropped my device and the battery connector came off!

I tried to solder it back on but it wouldn't work anymore :-(
I've ordered a new Tiny BLE, but for now I'm using the other battery socket on the Tiny BLE. Everything still works, but I have to charge my LiPo using a separate LiPo battery charger.

I received my low, mid and high sensitivity magnetic sensors, and my low, mid and high strength magnets, to cover a very wide range of magnetic sensitivity. A whole smorgasbord of magnets and magnetic sensors!

(Notice the 4 different tiny magnets on the bottom of the photo)

First I grabbed the most sensitive magnetic sensor, and soldered it onto a random SMD prototyping board, and tested if I could detect the metal pole that is already part of the exerciser. Unfortunately the pole wasn't magnetic at all, so I knew I'd need to glue an actual magnet to the end of the metal pole instead.

So then I did a quick test of a random magnet and a random sensor, and I could definitely detect that! I tried various combinations, and found that if I use a high strength magnet or a highly sensitive sensor, then unfortunately each finger effects multiple sensors. Or if I use a weak magnet or a weak sensor, there isn't any interference between fingers but I can't detect the whole range of movement, meaning I'd have to squeeze to the end before it gets detected. But if I use just the right amount of magnetic strength and sensitivity, it works perfectly! Detection of the full motion range and with interference between fingers! The best results were by using this combination:

• 1.9mm SMCO5 cylindrical magnet (https://www.digikey.com.au/products/en/sensors-transducers/magnets-multi-purpose/511)
• 90mV/mT DRV5053V analog linear magnetic sensor (https://www.digikey.com.au/products/en?keywords=296-38460-1-nd)

After I knew which magnet and which sensor to use, it was time to actually install one into each finger of the exerciser!

I occasionally play guitar, and when I'm not on the guitar I'm probably eating or on a computer. I bought a guitarist finger exerciser a while ago for about $5 in a music shop, that seemed useful to reduce RSI / Carpal Tunnel syndrome.
One day I was on the couch watching Netflix on my TV-connected laptop, with a mini keyboard + trackpad on my lap and my finger exerciser in the other hand, and I thought: can I measure these finger squeezes and use it to control my computer? Surely I can measure something, either a digital open/closed signal or possibly even an analog distance. When I opened up the exerciser I saw that each finger is made with a thin metal pole that's surrounded by a metal spring, that presses against the plastic surroundings.

The metal spring is the part that strengthens your finger muscles, and the metal pole would be to make sure it only moves straight along the axis.

When you squeeze a finger, the metal pole comes closer to the end of the tiny plastic cylinder, but it stays inside. So I tried finding a surface-mount Infrared LED and optical sensor that is tiny enough to send a reflection into the hole and measure the reflectance. But I realized it's such a small hole (around 3mm diameter) that it would be very tough to get it to work, and it would have a relatively large amount of electronics on the end to fit all the parts, so it might become a little uncomfortable to hold, but it probably could be done with some effort.

Then I thought that maybe I'm lucky and the metal pole is magnetic and the metal spring isn't, and then I'd possibly be able to measure the metal pole using a magnetic sensor. I didn't have enough experience with magnets and magnetic sensors to know if I'd need an ultra sensitive sensor or a fairly weak sensor, or whether the pole was magnetic at all.

So I decided to read the spec sheet of roughly 20 different types of magnetic sensors, until I realized that while many of these sensors measure a yes/no presence of a magnet, or they measure a changing amount of magnetism (such as an electric guitar pickup), I want a magnetic sensor that measures a stationary magnet and the sensor sends an analog output. Luckily I found a sensor that does this and comes in multiple sensitivity ranges. I still wasn't sure if the metal pole is magnetic or if I need to glue a weak magnet or a strong magnet to the end of the pole. I also didn't know if the magnetism of one finger would interfere with the other sensors, since each finger is quite close apart. So I decided to buy several tiny SMD magnetic sensors of low, medium and high sensitivity, as well as several tiny magnets of low, medium and high strength. I figured there has to be some combination that works pretty well!