EWNA Hackathon Team Trash

1:30PM

We are reaching the end of the allocated time we have for this project. We have chosen a model and tuned our outputs to vary a bit depending on the parameters we are reading from the image so we can get a slightly more accurate reading/prediction of the trash we are sorting.

We have had Ruize and Isaac working on the model's vision and determination of class of trash that we are presenting it with. They also worked on getting the python code files to successfully launch the camera and call on the bridge functions to start the communication channel between the MPU and MCU.

Cory and Matthew worked on the hardware side of things by setting up the physical connection between the board and external components as well as setting up our conveyer belt and camera to get the best angle we could find to look over the trash we were putting onto the display.

Currently we are done with all the individual components of the project and it is just the connection of the pieces that we are currently working on.

4:54 PM - A few hours after the initial log, the separate pieces of our project have moved forward.

On the software side, Isaac and Ruize are testing different Roboflow datasets between an Identification and Classification model.  They ran the training for our YOLO model, and the initial validation metrics partially work. Matthew decided that downloading and combining multiple datasets may help with accuracy.  While waiting, we looked into other potential methods for recognition and came across Vision Language Models such as Moondream and SmolVLM.  Given our hardware specifications, it was decided that we would stick with the CNN.  During training, Ruize determined the model would take too much time to train with extra datasets so we’ve decided on a pre-trained model with slight modifications on accuracy readings.  

Meanwhile, Cory is finalizing the serial communication between our Python environment and the Arduino UNO Q with the OLED graphic display.  The screen did not display anything for a while until Cory burned his finger on the screen.  A minute later, we were able to finally display a message.

Our next step will be tuning the confidence levels for our readings before flashing the trained model onto the Arduino.

Once the clock hit 11:05, our team immediately opened up the box and all read the challenge. After reading the prompt, we all knew what we needed to do and that was to begin researching how to accomplish this challenge that was put in front of us.

Our team:
Cory: Started to build the conveyor belt contraption.

Isaac, Matthew, Ruize: Looked into different methods to achieve decent accuracy for the machine vision detection.

Ruize: Setup the Arduino UNO Q, configuring the IDE and downloading all the necessary resources.

While researching about the different models we could implement to solve this challenge, some of the previous knowledge from past classes had been brought up. Whether or not we used a  (CNN) or a standard LLM was a debated discussion within our team. We also were thinking on whether or not it would be a good idea to divide the R&D into two implementations and exploring whether or not it would be beneficial given the limited time-frame had. 

We opted to go with one path, the CNN approach using Ultralytic's You Only Look Once (YOLO) vision model. By using that vision model in conjunction with different datasets from Roboflow, Ruize, Matthew, and Isaac was successful in setting up a working detection script in Python, using the webcam provided by the challenge to detect everyday items.

On the other hand, Cory was looking at ways to improve on the conveyor belt design and placement of the webcam for the optimal viewing angle and extraction time. While also working on the logistics for the team.

Currently, the team is still in processing of getting a good dataset and testing the of the model, while for hardware, designs are being sketched up and implemented.