I have been inspired over the last few weeks by Fabio's project with Edge Impulse and how the Tracer platform can be used to leverage the ESP32's processing power to run neural network classifiers of activities and gestures. In this example, I will go through the steps needed to train your own neural network and show it performing live classification with data streamed directly from the board. 

I wrote an Arduino sketch that would spit out the 3-axis accelerometer and 3-axis gyrometer at a frequency of 100Hz. The experiment I had in mind was basically to use Tracer as a classifier that could tell if it was moving horizontally, vertically, "caterpillar" (move in a jagged way horizontally), and waving. A dataset of 10s was collected for each of the movements.

Once I had an idea of what I wanted to build, I followed the steps shown in the video tutorial from Edge Impulse in getting a model trained up as well as a k-means-based anomaly detector.

The initial model that was trained showed inaccuracies when it came to classifying a wave and that is due to several reasons but I think it was primarily due to not having enough data. The table below shows the confusion matrix for the first training cycle with the original dataset.

Following that, I wanted to see if collecting more data would improve so I spent another 10 seconds collecting data for wave, and it improved slightly. I think more data in each category should improve the neural network's specificity and sensitivity. The accuracy improved across the board while the error rates also dropped.

Here is a short video example of how the live classification tool is being used on the data that is streaming directly from Tracer over USB (UART). The next step for me is to deploy a real-time classifier for tennis shots that will run on the ESP32 and stream the results over BLE.

And again as before, really happy to hear any feedback, ideas or suggestions!