A detection model can only be as accurate as the data it learns from, so we labelled our training images ourselves. We hand-labelled more than 750 images, drawing a bounding box around each object in every frame. It took our team about 2–3 hours of careful, repetitive work, but hand-labelling meant every box sat tightly around its object. That precision is what the model needs to learn exactly where the marker is.

The model now detects the marker in real time and gives a confidence score for each detection, showing how sure it is that the object is a marker. It picks out the Crayola marker reliably from both a front and a side angle. It also recognises the robot arm itself, so the system can tell the marker apart from the arm reaching for it. We also compared these results with a marker-only model (the thin "m-only" boxes).

The marker is detected with 0.84 confidence and the arm with 0.51. The marker-only model finds the marker at 0.76.

The marker is detected with 0.83 confidence and the arm with 0.75. The marker-only model finds the marker at 0.75.