The attached YouTube Video was used to construct the robot and move to next stage which was interfacing sensors and actuators through firmware.

For starters, we referred

1. Random Nerd Tutorials Article (Link: https://randomnerdtutorials.com/esp32-servo-motor-web-server-arduino-ide/) for Servo Motor interfacing and then developed logic further to control four such motors according to the problem statement.
2. Electronic Wings Article (Link: https://www.electronicwings.com/esp32/analog-joystick-interfacing-with-esp32) for Joystick interfacing and then developed logic further to control two joysticks with left/right alignment according to the problem statement.

The combined algorithm was developed by the firmware wizards in the team which brought the setup in the current state.

The Arduino UNO Q is the "brain" of our arm. It has two processors on one board: a Qualcomm QRB2210 running Debian Linux, and an STM32 microcontroller. We run the whole perception and planning stack on the Linux side, fully offline. Real-time servo control stays on the ESP32 Max V1.0 from Step 2, which the UNO Q talks to over USB serial.

Hardware connections:

1. The UNO Q has only a single multi-function USB-C port, so we used a USB hub with a power delivery input (via USB-C). [hub model] 
2. The Logitech C270 webcam plugs into the hub. It's mounted overhead, looking down at the work surface.
3. The ESP32 plugs into the hub with its USB cable and shows up as a serial port (usually /dev/ttyUSB0).

First boot and access:

1. The UNO Q comes preloaded with a Debian-based operating system, so no image flashing was needed.
2. We installed Arduino App Lab on a laptop. App Lab takes care of connecting the UNO Q to WiFi and configuring SSH. We also used it to update the board.
3. After that we worked from a terminal over SSH:.
   ssh [user]@<UNO-Q-IP>
4. We checked that the camera and ESP32 were detected:
   sudo apt update
   sudo apt install -y git python3-venv python3-pip v4l-utils
   lsusb
   v4l2-ctl --list-devices
   ls /dev/ttyUSB* /dev/ttyACM*
5. We gave our user permission to use serial ports (then logged out and back in):
   sudo usermod -aG dialout $USER
   We chose to run our code as a plain Python program on Debian rather than as an App Lab app. That keeps every module testable on its own, with no extra framework (and no ROS 2).

All the Python code lives in the files section (.py files with _ac at the end of filename) of our project page. It runs as one process on the UNO Q's Linux side.

Environment setup (on the UNO Q):

cd arm_robotic/arm_control
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Libraries installed in that virtual environment:

• opencv-contrib-python (4.8 or newer) handles camera capture, colour detection, background subtraction and the pixel-to-millimetre mapping. We need the "contrib" build rather than plain opencv-python, because path extraction uses cv2.ximgproc.thinning.
• numpy handles the image and array maths.
• pyserial handles USB serial communication with the ESP32 at 115200 baud.
• tflite-runtime is optional. It's only needed if obstacle detection is switched to a trained model (see below).

On-device intelligence: everything runs locally on the UNO Q with no network connection. The current version uses classical computer vision in OpenCV:

• The marker is found by its colour (HSV thresholding).
• The drawn path is reduced to a one-pixel-wide line and walked from end to end to get an ordered list of waypoints.
• Obstacles are detected by comparing each live frame against a saved photo of the empty workspace.

We chose classical vision because it needs no training data and behaves predictably during a demo. The obstacle module also has a second, drop-in backend that runs a TensorFlow Lite object-detection model (for example, one trained with Edge Impulse). Changing OBSTACLE_BACKEND in config.py swaps between them without touching any other code.

What each file does:

• config.py holds every tunable value: camera, serial port, colour ranges, arm dimensions, step size.
• camera.py is a small wrapper that reads frames from the C270.
• calibrate.py is a one-time setup tool. You click 4 known points on the table and type their real positions in mm, which saves homography.json. It also saves a photo of the empty workspace for obstacle detection.
• tune_hsv.py gives you sliders for finding the colour ranges of the marker and the path ink under the actual venue lighting.
• homography.py converts camera pixels to table millimetres and back.
• marker.py finds the marker by colour.
• path_trace.py turns the drawn path into an ordered list of waypoints, starting from the end nearest the marker.
• obstacle.py contains both obstacle detectors (classical and TFLite) behind one interface.
• kinematics.py is the inverse kinematics solver. It converts a target X/Y/Z position into base, shoulder and elbow angles for our arm (link lengths 78 mm and 50 mm).
• arm_link.py sends MOVE/HOME/PING/RESUME commands to the ESP32. It also listens in the background for kill-switch events.
• planner.py is the "Guarded Move" loop. The arm moves 5 mm at a time and re-checks the camera before every step. If something appears in the way, it steers 30 mm around it.
• state_machine.py runs the full task: find the marker, pick it up, trace the path, put the marker back, return home.
• main.py is the entry point that connects all of the above.

Bring-up order (we tested each part on its own before running the whole thing):

1. Camera check:
   python3 -c "from camera import Camera; import config; c=Camera(config.CAMERA_DEVICE, config.CAMERA_WIDTH, config.CAMERA_HEIGHT, config.CAMERA_FPS); print(c.read().shape)"
2. Calibrate with the camera in its final position: python3 calibrate.py
3. Tune the colours for the marker and the path ink: python3 tune_hsv.py, then paste the printed values into config.py. 
4. Test marker tracking, path detection and obstacle detection one at a time with live frames. 
5. Test the ESP32 link from Python: ping, a single move, and the kill switch. 
6. Check the inverse kinematics against a few hand-measured points. 
7. Run the full task:
   python3 main.py --port /dev/ttyUSB0 --camera 0

calibrate.py and tune_hsv.py open OpenCV windows, so run them with a monitor attached to the UNO Q or over SSH with display forwarding (ssh -X).