Introduction

Chess clocks have been used in competitive chess games since 1883. There are few variations but the main idea how they work didn’t change for over a century. Apart from time controls configuration at the start of the game the only interaction with chess clock is a player pressing a button after each move to stop their own clock and start the opponent’s. Novice players often forget to do that. Given advancements in computer vision it is time to create a hand free chess clock. And not in traditional hands free devices sense, meaning that it is voice controlled. Idea it to use camera to recognise when move is finished and automatically stop current player clock and resume opponent’s clock. Thus freeing players from any mandatory interaction with a clock except initial setup. An intelligent clock could actually understand chess positions, annotate games, detect illegal moves, stream online, give advice during training and much more.

Here is a quick demonstration what clock can do already:

Mechanical design

Frame is made mostly from eitech construction set parts, rubber feet are from old printer rollers, camera mounting plate is 3D printed.

Voice commands

Clock is controlled by voice commands. Audio is captured using Advanced Linux Sound Architecture (ALSA) library at 16kHz sampling rate. Silero Voice Activity Detector (VAD) is used to find start and end of the speech. VAD is using Open Neural Network Exchange (ONNX) Runtime. Detected speech is recognised using whisper.cpp library which is a plain C/C++ implementation without dependencies of OpenAI Whisper model. whisper.cpp library is compiled with cuBLAS for CUDA support. Small English only Whisper model is used. Inference time varies from under a second to several seconds depending on the input. This is real-time enough for clock application given it receives commands quite infrequently. Transcribed text is pattern matched (using regular expressions) against expected commands. If match is found relevant command is executed. Supported commands:

• start x minute(s) game [with y second increment] - starts a new game. All chess pieces must be at their initial squares. After starting game video camera will observe the board and automatically switch relevant clock after each move.
• stop the game - will stop the clock.
• continue game - will resume clock.
• shutdown - will halt computer running the clock. Handy when you don't have keyboard or remote terminal for a safe shutdown. /etc/sudoers file must be modified accordingly.
• please tell best move - will use external chess engine to evaluate current position and will tell what it thinks the best move is.
• what is worst move - will use external chess engine to evaluate current position and will tell what it thinks the worst move is.
• who is winning - will use external chess engine to evaluate current position and will tell who is winning.

Slight variations of command wording will work too.

Voice output

Hands Free Chess Clock provides voice feedback after commands, chess moves and when game ends. It can also tell opening name from Lichess opening database. Voice audio is generated on the fly using Piper text to speech (TTS) engine.

Chess board detection

Open Source Computer Vision Library (OpenCV) is used to detect chess board. Software doesn't attempt to detect board in arbitrary orientation or with arbitrary initial chess position. Assumption is made that when start new game command is issued chess pieces will be at their initial positions, board will be in the view of the camera and aligned with the chess clock.

Wooden chess board shown in the video above is my first chess board I ever played on. It is over 30 years old and is quite worn. This makes detection problem more challenging because of extra visual artefacts on the board. On the other hand it is good for tuning vision algorithms...