11:00AM - 12:00PM
Understanding the Challenge

The first step in this process was establishing our system requirements. Interpreting the challenge instructions, what should our final product look like? We decided to utilize all the given hardware components while imagining a real use case for hand-gesture recognition in an industrial workplace.

We describe a scenario of a factory worker needing to assess condition of a box (or skip, if validation isn't necessary). The system will recognize the set of three gestures and respond within 200ms with visual and audio cues. 

12:00PM - 1:00PM

Dividing Responsibilities

We decided to split the responsibilities into two groups: model development and then hardware integration. 

First, we needed to fine-tune a model to read three gestures: thumbs up, palm, and skip. This required collecting significant amounts of test data and labeling appropriately. This work would continue into Day 2. 

Second, we need to develop a system that uses this YOLO computer vision model and uses the Arduino components as a visual & audio interface.

1:00PM - 3:00PM

Adopting Open Source YOLO Model and Designing Features

We collected many images in different light settings, backgrounds, and positions to capture a comprehensive dataset for labeling.

Simultaneously, we were designing how requirements developed into features. For example, the Pixels module will be used as pitch/note indicators corresponding to the frequencies being played on the Buzzer. The on-board LED matrix will be used as a primary indicator of gesture recognition.

3:00PM - 5:00PM

Resolving Network Issues

Once we finally got the model on the board, we discovered it was impossible to get the camera connected to the Uno Q. For hours, we spent our time debugging and researching possible solutions online. Finally around 5pm, we resolved the camera issues and were able to start testing the system.

5:00PM - 6:00PM
For the last hour we were still in the process of developing the system logic and did not have much progress.