Demo

I might make a video later but for now a shot gif will have to do. I've been using it to control music, as shown above, but it's capable of acting as a full keyboard as well.

General Specifications

Hardware

The BMI160 ring collects IMU data which is processed via the wrist-mounted Seeed XIAO nRF52840. The whole thing is powered by an 80 mAh lipo stolen from broken earbuds. Five wires are run between: two for power, two for I2C, and one for the button. Wrist mounting is done via any 32mm watch strap and prevents an excessively bulky ring.

Data Collection and Processing

The shapes are "drawn" in two dimensions using numerical integration of the pitch (y) and yaw (x) axis, similar to how a WII remote moves around. Data is sampled periodically and stored in buffers capped at 500 raw points. Once the gesture is complete, a subdivision algorithm transforms the shape into a fixed number of evenly-spaced points, which are then normalized (adjusting for the aspect ratio, thanks Zack Freedman) between 0 and 1.

The Encoder

Normalized shapes are processed through an extremely complex network consisting of a single convolutional layer, flattening, and then a single fully connected layer. Written in Tensorflow then converted to TFLite with default optimizations. I'll likely do a deeper dive into this later.

The VDB

Shapes are collected via a computer program through multiple samples, normally 10 but more or less can be used depending on accuracy. Each shape is stored as a mean, standard deviation, and associated action. This takes the form of several arrays of floats for the first two and an HID action code for the last one. 

The Detection

After receiving the encoded shape, the detection algorithm loops through every shape in the VDB and runs a Z test to check if the sample is part of that shape's "population." This is best for appeasing the math people, but a modified z-score formula with a squared numerator tends to work far better. Due to the relatively low number of shapes normally in use (<100), no fancy optimizations were done.