We realised our first design was ambitious. While tracing, the arm would check the camera every ~20 mm, and if something new had blocked the path, it would re-plan just the remaining route and keep going. We built it and it worked in testing, but the more we thought about the actual challenge, the more we questioned whether we needed it.

The organisers clarified that during the demo, the line and obstacle are placed before the run starts and stay put. With a still scene, watching the camera mid-trace adds extra steps, extra processing and extra ways for something to go wrong, with no real benefit.

So we simplified. Now the arm takes one look at the scene and plans everything up front:

1. Finds the marker, the line and the obstacle in a single camera frame. The arm is recognised too, so it's never mistaken for an obstacle.
2. Plans the line, including any detour around the obstacle.
3. Stitches everything into one complete route: marker → start of the line → along the line → end of the line → back to the marker.
4. Saves the full plan to a file, with every point labelled as drawing, detour, lift-over or travel, plus an overlay image so we can check it before the arm moves.

Then the arm simply executes the plan from start to finish. Sometimes the best engineering decision is knowing what to take out!

The full planned route overlaid on the camera view before the arm moves.