Smiling Stack-chan, synchronized with my face!

It can follow my face orientation, eye/mouth opening and smiling.

I used Google MediaPipe for face tracking. MediaPipe extracts facial contours and meshes with very high accuracy. I used those meshes to calculate the orientation of the face and the opening and closing of the eyes and mouth.

I used WebSocket to send information to Stack-chan. The Moddable framework includes a WebSocket module. It's easy to use and fast enough, about 20 fps in the case of the above video.

Face synchronization adds a powerful capability to Stack-chan for many use-cases such as teleexistence avatar robot.