PionEar: Making Roads Safer for Deaf Drivers

I'm sharing a video where I'll introduce my project in more detail, discuss the motivations behind starting it, and share my plans for the future.

As I have indicated in my project entry video, I plan to turn the PionEar sensor into mass production. Of course, this can be done after the careful evaluation of the “market size” and annual sell volume estimation. This process is currently ongoing. However, I would like to demonstrate possible scenario in what way the PionEar sensor can be produced in larger quantities.

First of all, it is necessary to realize that the current proof of concept prototype - PoC is mostly built on development boards. This is good for building the first concept, performing various tests and evaluations, or even providing it to the first customers to receive valuable feedback. Using the evaluation boards saves a lot of time that would be needed for the development of custom-made designs. However, it is not very effective in terms of cost and size/fit factor. For this reason, I plan to develop a custom-made PCB where all the electronics will be placed. This will greatly simplify the sensor assembly process as the wiring will be minimized to a few connections (Battery, solar panel) With a custom-made PCB one can also expect cost savings on the component/material level.

The next part to consider is the manufacturing process of the plastic housing. Currently, the whole housing is made with a 3D printer. The 3D printing process is again very suitable for prototyping and product iteration. But with the current ease of access to 3D printing technologies and significant price reduction, it can preferably be used for low production volume as well. I plan to use 3D printed parts also for production in volumes of several hundred up to one thousand pieces per year. According to my investigation up to this volume, 3D printing will be cheaper compared to injection molding. I also plan to greatly simplify the process of creating the light diffuser for the illuminated logo. In larger volumes, I will very probably use dual-color 3D print (partially translucent filament for logo infill) instead of pouring the SLA epoxy as described in the build instructions.

In terms of cost, I have evaluated the material cost for the current prototype and compared it with an estimated cost for a version with a custom PCB (calculated for 1000 pcs batch). As can be seen from the tables below, the cost has been reduced to approximately 35% of the initial value.

In order to complete the cost of the product, the labor costs of the production process and the costs of transport and packaging must also be calculated. The final cost of the sensor is evaluated in the following table. If you're wondering how I derived these figures, they're based on a combination of my experience in electronics manufacturing, Prusa 3D Printing Price Calculator and a discussion with ChatGPT 😊

I've already shared a couple of videos demonstrating how I test the PionEar sensor and its machine-learning model performance in a car. I've captured some nice footage of it detecting ambulance sirens. Before I actually get in the car and spend time driving, I first run tests using a PC and YouTube. This initial testing provides valuable insights into whether it's worth testing in a real-world environment. Additionally, I want to showcase that the current machine-learning model is quite resilient to various traffic noises.

During the hot summer days, I realized that the standard PLA material might be problematic for use in car interiors. The temperatures can easily reach more than 70°C or even 80°C. For this reason, I decided to conduct a simple test in a real environment. One day, when the outside temperature reached 35°C, I left a couple of sensor housings on the car dashboard. The car was oriented in such a way that the dashboard was fully exposed to sunlight. In this test, I compared the currently used PLA materials with one of the PET-G samples that I had bought, as I liked its color as well. I was hoping that this material sample might be the right choice for this application.

After approximately 2-3 hours of sunlight exposure, I checked the results. As expected, the PLA samples became very soft and lost their shape after I squeezed them in my hand. The PET-G material (Spectrum PET-G Stardust Blue) remained rigid, and I could not deform it after pressing it with my fingers. This test showed that PET-G filaments are definitely a better choice for such applications. I cannot exclude the possibility of repeating this test under more extreme conditions and perhaps choosing a different material again. But for now, it seems that PETG filaments are good enough for most cases.

It is also worth mentioning that since the last time, I have updated the mechanical model of the housing. Now it is ready for mounting the solar panel. The whole box is now slightly bigger in order to fit the panel on the top of the housing. The build instructions have been updated accordingly.

In this latest update, I'm excited to introduce a new firmware that prioritizes energy efficiency. It leverages the fact that the TinyML board has a neural network accelerator (NDP101) and host processor (ATSAMD21G18) as two separate devices. The NDP101 can process the input audio with extremely low power consumption. While NDP101 is processing the audio, the host MCU can be in sleep mode until the moment the NDP101 recognizes the desired sound pattern. If this happens, the NDP101 wakes up the host MCU via a dedicated interrupt pin. As the host MCU takes the largest portion of energy, this approach will dramatically reduce total power consumption.

Before the low-power mode was implemented the total current draw was about 15mA. It's now been impressively reduced to under 2mA ! This current is low enough to power the whole device with a small solar panel (keep the battery charged). FW is available on GitHub – Please follow the readme file for further instructions. You can also watch the demonstration video on YT. 

After updating the ML model, I conducted more comprehensive road tests. I captured some compelling footage that demonstrates how the sensor performs under real-world conditions. Particularly, the first scenario is a major concern for deaf drivers.

Today, I want to provide an update regarding the ML model which I have updated. First of all, I will describe an issue with the old model. During the testing I noticed, that detection sensitivity is very poor when I drive a car at moderate/high speed. After analyzing this problem I have found, that this is caused by additional low-frequency noise generated by the car (engine, tire rolling, etc.), that masks the sound of sirens. 

First I wanted to implement any kind of high-frequency/band-pass filter to analyze and evaluate just the frequency band where a siren typically emits a sound. But I found, that there is no easy way to make it with TinyML board or Edge Impulse studio.

After this, I decided to create a new ML model that will "learn", that low-frequency noise in a car cabin together with siren sound should result in positive class detection. So I turned my current prototype into a sound recorder and recorded several hours of driving sound. After this, I mixed this with already existing siren sounds.

I also used a new dataset for siren sounds: sireNNet. This Dataset seems to have better quality as the recordings are cleaner and do not contain any other (often very strange) type of sirens, that were present in the previous dataset. Even though this dataset is much smaller, the observed result in the real driving conditions was much better.

For a better understanding of how I created the current dataset, please see the block diagram “Dataset_8_2023_diagram.pdf” which I share together with other files. I believe that this might be useful also for other people when trying to build any ML model that should run in a real environment.

In the block diagram, you'll see that I use varying amplification levels for certain classes. I've discovered that this greatly helps the model in achieving a higher level of abstraction when recognizing sounds. Without this adjustment, the model frequently identified different classes (sounds) based on their average sound level rather than their frequency content.

My current dataset is again shared on the Edge Impulse Platform here:

https://studio.edgeimpulse.com/studio/267982

Despite the longer inactivity on my project page I have been still actively working on the project. One of the issues that I've been trying to figure out is the Li-pol battery safety concern.

It's well-known that temperatures inside a car can soar, especially during the summer. Naturally, this raises concerns about whether the battery inside a sensor could overheat, leading to potential fires or explosions.

After delving into the subject, here's what I've found: Standard lithium-polymer (Li-Po) batteries typically have a temperature range of up to 60°C for discharging and up to 45°C for charging. It's not hard to imagine a car's interior exceeding these temperatures. However, even if the temperature goes above these limits, a Li-Po cell remains stable. It's only when you hit the 130-150°C range that the cell becomes thermally unstable, which could result in a thermal runaway where flaming gases are released. But, realistically, reaching such extreme temperatures inside a car is quite improbable. The manufacturer's temperature guidelines seem to be more about minimizing cell degradation than they are about safety.

Nevertheless, in the interest of maximizing safety, I looked into high-temperature batteries. Although there aren't many options on the market, I did discover that Zoncell produces high-temperature cells. These have a temperature range that extends up to 80°C for discharging (and 45°C for charging). Some might point out that 45°C as a charging limit might not be sufficient, especially since I plan to incorporate solar charging. It's worth noting that the lower charging temperature limit is primarily because of self-heating caused by the charging current. This is especially true for charging currents close to 1C or higher, where the heating effect is significant. However, based on my recent tests, solar panels can only deliver a maximum of 30mA, which is less than 0.1C. Such a low charging current won't result in substantial self-heating, so I'm confident that the true temperature limit is much closer to the 80°C discharge limit.

So I ordered a few samples of Zoncell LP602530T (datasheet enclosed in shared files) and will consider their further use in the PionEar sensor.

In the next iteration, I would like to integrate a solar panel into the PionEar sensor to avoid the need for charging. I did some research for suitable low-power solar charger modules with MPTT functionality. I discovered AMELION from @Jasper Sikken. In combination with IXYS SLMD960H09L solar panel, it seemed to be a good starting point. AMELION module is a great solution for low-power applications. It can still provide some current even in the indoor environment. I could measure the following charging currents:

Indoor: 20 to 50 uA

Outdoor (cloudy/shadow): 2 to 5 mA

Outdoor sunlight: 25 to 30 mA

These numbers seem to be promising. I think that behind the car's front window, there should be plenty of light to keep the battery charged - unless you park in the garage.

Usually, as a first test for audio event detection, I use YouTube as a source for various audible events. But the real-life test is irreplaceable. In the case of PionEar sensor, it was a bit tricky to catch the ambulance with sirens on and have a camera ready. Last time I spent 2 hours driving around the nearest hospital and wasn't able to meet any car with sirens :-) Today, on my way to work I was lucky and made a shot.