The central conceit behind the mimic robotic arm is the wireless control of the robotic arm using muscle sensors (EMGs). Before getting initial any of the details, let's show it working.

Now that you've seen it working, the question is how?

Despite looking somewhat complicated - looking at you breadboard - the guts of the project are actually quite simple. Let's break it down into the mechanics, electronics, and software components.

Mechanics:

The basic construction of the arm is similar to any of a thousand CAD models you can find on the internet for a toy robotic arm. We happened to use this model, but the choice is totally optional.

From there it was a simple matter of 3D printing and assembly to get a basic platform to work with. One thing to note is the addition of a base plate. That's really key to stop it from tipping over.

With the easy mechanics out of the way we can move on to the electronics:

Electronics:

On the surface, the electronics behind the EMGs look very complicated. I mean just look at this monstrosity of a breadboard!

But really this is just once circuit duplicated multiple times for each of the motors. In reality, the actual mechanism is rather simple.

There are only two stages. First, an op-amp based amplification circuit and a level shifter scales the signal into the 0-3.3V range of the ESP32 chip.

Second, a series of high pass filters get rid of the low frequency noise. As you can see from this oscilloscope reading, there is a ton of low frequency noise when a muscle adjacent to the one you are measuring gets flexed. By getting rid of this noise you prevent neighboring probes from interfering with one another.

Once everything has been properly filtered and amplified it can be sent directly to the analog digital converter pins of the ESP32 board.

Of course, you don't have to use a bread board either. Eventually we replaced it with a nice looking custom PCB.

One other thing worth mentioning here is that there is also simpler setup for joystick control.  They don't need filtering/amplifying so the leads can hook directly to the analog to digital pins on a second ESP32. This allows for control of the arm without the hassle of putting on a bunch of probes. It was very useful for testing.

Regardless of whether it is the joysticks or EMGs, once the signal is on the ADC pins it is time for software to take over.

Software:

The software aspect of the project is where things really start to get complicated. Broadly speaking it can be split into four different stages. 

+ Signal Processin

The first stage is signal processing. In this stage the client EMG on the sensor interprets the incoming data and prepares an instruction for the servos based on that interpretation. To understand how, it is first necessary to grasp the concept of velocity control. In a velocity control scheme the servos are not explicitly told to go to any particular angle. Instead they are told to increment, decrement, or stay still for each clock cycle. Over multiple clock cycles if the instruction is increment forward, for instance, the arm appears to have an outwardly smooth forward motion. Velocity control is at the heart of the signal processing step. For each motor on the arm, the client reads the incoming data and decides if it is above some threshold. If it is, then the client knows that the corresponding motor needs to increment. If the data is below the threshold that corresponds to a rest state. To move backwards there are extra sensors on the arm that measure the extension of the arm. When those exceed the threshold the client know the motor needs to reverse.

Once the client has decided what each motor needs to do, it packages it all up in the JSON format. The signal is now officially processed.

+ Message Sending

This is where the second phase comes in: message sending. In the message sending phase the sensor based clients send their JSON formatted strings to the server based on the arm. This...