Things used in this project:

Hardware components

Arduino Nicla Sense ME×1
Bosch BHI260 sensor

Software apps and online services

Neuton Tiny ML
Arduino IDE

Intro

Despite the incredible variety of wearable devices today, most of the AI features come down to merely defining very simple actions. They determine whether one is running or sleeping, they count the steps, and they determine if one falls.

It seems to me that we are ready for much more complex analytics from our devices. What if they could recognize more specific, similar, and complex activities? It would open up the never-before-seen possibility of creating a great many new interesting applications and devices.

However, recognition of more complex activities requires large and complex neural networks which require a great amount of computation. This would obviously quickly drain the battery of the IoT device where this neural network would be embedded. The majority of wearable devices should operate in the always-on mode, which places serious demands on the energy efficiency of such solutions.

How can we solve this challenge? In my project, I want to show an example of how you can significantly increase the complexity of intelligence functionality, and determine what a wearable device can recognize with a very small and accurate neural network.

In my project, you will find a detailed guide on how to implement Multiple Daily Activities recognition using the Nicla Sense ME.

If you have Nicla Sense ME, you can try my pre-trained model in your experiment/project/classes.

You also can participate in my practical webinar on April 11 at 5 pm CET. Where you can watch how to create this solution from leaders from Neuton.AI, Arduino and Bosch. Don't miss this opportunity to learn from experts directly.

Our test results show that the model perfectly recognizes the movements of 30 participants whose data was not used for training the model.

Following this tutorial, you also can easily make your own experiment and train your neural network on your data to recognize the same or different movements.

We believe that the ability to recognize an extremely complex and similar action using small neural networks opens up a new era for always-on devices.

Below, you will find a full tutorial on how to reproduce this experiment, including data collection, model training, model embedding and inference. In addition, we have created a public repository with a pre-trained model that has demonstrated good generalization on new users. It is a precompiled model archive that you may just download and embed into your Nicla Sense ME with a few clicks of a mouse and start testing.

https://github.com/NeutonTinyML/hand-activity-recognition

Data Collection with Nicla Sense ME

For this case, I’ve selected 5 types of activities (for your experiment, you may choose other activities):

• Washing hands
• Brushing teeth
• Clapping
• Brushing hair
• Random activity (negative class)

For each activity type, I collected sensor data with Nicla Sense ME for 10 minutes non-stop. Data was collected from 7 different people from 7 different locations. It was quite a challenging, yet fun process because, for proper data collection, it was undesirable to interrupt. Here’s some advice for those who plan to re-conduct my experiment: before collecting data for another activity, get some rest! However, if 10 minutes of constant hair brushing is too much, you can do two 5-minute data collection streams and then concatenate the two files.

For the negative class, we collected 60 minutes of data. There’s no need to do anything special this time; just get on with your everyday activities. Type on the keyboard, operate a mouse, answer calls, drink tea, or do whatever you want. Just be sure you don’t sit completely idle with your hands still during negative class data collection. The negative class is a subset of everything else excluding the 4 classes outlined above.

Steps:

Now let...