What components I used ?

  There are three main components: ESP CAM, Mini Pan-Tilt Kit, and FTDI. ESP CAM is a feature-rich MCU. It integrates WIFI and Bluetooth. The camera model is OV2640. FTDI gives ESP32CAM power and transmit and receive data. Mini Pan kit has two servo motor. It could rotate 180° horizontal and 150° vertical. The angle command is given by the ESP32 MCU chip. The total price is about $30.

How to track human pose?

There are many ways to handle human pose estimation. It is a popular topic now. I use PoseNet with tensorflow.js in this project. It could process the images directly using JavaScript. It gives 17 confident score and positions of key points. Confident score is used to evaluate how much we trust this point. The skeleton of the human body could be draw real-time in the webpage.

How to rotate servo motor?

After detecting position in canvas, we should give commands back to put face or body in the middle. I test different ways to track, such as using average of position or real-time position. The robustness is not good and always have a delay to track.

Finally, I use the relationship of size of canvas and angle. For example, it we track the nose, and the width of canvas is 640, the angle of the full width is around 50 degree. We could derive a function to rectify the nose position to 320.

How to evaluate human pose?

There are different ways to evaluate human pose, such as deep network. I use a traditional way.  After detecting position in canvas, we test the position and angle of leg or arm. For example, the wrist position is higher than eyes, and the distance of ankles is larger than distances of shoulder when doing a jumping jack. 

Final, here is a video of the webpage I design:

To see more details: please visit https://github.com/JiabinNU/PoseTracker