Multi-Model AI-Based Mechanical Anomaly Detector

Description

Mechanical anomaly detection is critical in autonomous manufacturing processes so as to prevent equipment failure, ameliorate the effects of expensive overhaul procedures on the production line, reduce arduous diagnostic workload, and improve workplace safety[1]. In light of recent developments towards the fourth industrial revolution (Industry 4.0)[2], many renowned companies focused on enhancing manufacturing and production processes by applying artificial intelligence in tandem with the Internet of Things for anomalous behavior detection. Although companies take different approaches, and each technique has specific strengths and weaknesses based on the applied manufacturing mechanisms, autonomous anomalous behavior detection enables businesses to preclude detrimental mechanical malfunctions that are challenging to detect manually by operators.

Details

Nevertheless, there are still a few grand challenges to overcome while applying mechanical anomaly detection to mass production operations, such as the scarcity of data sources leading to false positives (or negatives) and time-consuming (or computationally expensive) machine learning methods[3]. Since every manufacturing system setup produces conflicting mechanical deviations, the optimal anomaly detection method should be deliberately specialized for the targeted setup, which minimizes false negatives and maintains exceptional precision. If the mechanical anomaly detection method is applied without proper tuning for interconnected manufacturing processes, the applied method cannot pinpoint the potential root cause of the detected mechanical anomaly. In that regard, inefficient anomaly detection methods still require operators to conduct manual inspections to diagnose the crux of the system failure.

After inspecting recent research papers on autonomous anomalous behavior detection, I noticed that there are very few appliances focusing on detecting mechanical deviations and diagnosing the root cause of the detected anomaly so as to provide operators with precise maintenance analysis to expedite the overhaul process. Therefore, I decided to develop a device to detect mechanical anomalies based on sound (via an onboard microphone), diagnose the root cause of the detected deviation via object detection, and then inform the user of the diagnosed root cause via SMS.

To be able to detect mechanical anomalies and diagnose the root cause efficiently, I decided to build two different neural network models — audio classification and image classification — and run them on separate development boards to avoid memory allocation issues, latency, and reduced model accuracy due to multi-sensor conflict.

Since FireBeetle 2 ESP32-S3 is a high-performance and budget-friendly IoT development board providing a built-in OV2640 camera, 16MB Flash, and 8MB PSRAM, I decided to utilize FireBeetle 2 ESP32-S3 to run the object detection model. To run the neural network model for audio classification, I decided to utilize Beetle ESP32-C3, which is an ultra-small-sized IoT development board based on a RISC-V single-core processor. Then, I connected a Fermion 2.0'' IPS TFT display to FireBeetle 2 ESP32-S3 in order to benefit from its built-in microSD card module while saving image samples and notify the user of the device status by showing feature-associated icons. To perform on-device audio classification, I connected a Fermion I2S MEMS microphone to Beetle ESP32-C3.

Even though this mechanical anomaly detector is composed of two separate development boards, I focused on enabling the user to access all interconnected device features (mostly via serial communication) within a single interface and get notified of the root cause predicted by two different neural network models — sound-based and image-based. Since I wanted to capitalize on smartphone features (e.g., Wi-Fi, BLE, microphone) to build a capable mechanical anomaly detector, I decided to develop an Android application from scratch with the MIT APP Inventor. As the user interface of the anomaly detector, the Android application can utilize the Wi-Fi network connection to obtain object detection model results with the resulting images from a web application, save audio samples via the built-in phone microphone, and communicate with Beetle ESP32-C3 over BLE so as to get audio-based model detection results and transmit commands for image sample collection.

As explained earlier, each manufacturing setup requires a unique approach to mechanical anomaly detection, especially for interconnected processes. Hence, I decided to build a basic frequency-controlled apparatus based on Arduino Mega to replicate mechanical anomalous behavior. I designed 3D parts to contain servo motors to move a timing belt system consisting of a GT2 60T pulley, a GT2 20T pulley, and a 6 mm belt. Since I utilized potentiometers...

Components

1 × PCBWay Custom PCB

1 × DFRobot FireBeetle 2 ESP32-S3

1 × DFRobot Beetle ESP32 - C3

1 × DFRobot Fermion: I2S MEMS Microphone

1 × DFRobot Fermion 2.0" IPS TFT LCD Display (320x240)

Build Instructions

Collapse

1

Step 1: Designing and soldering the Iron Giant-inspired PCB

Before prototyping my Iron Giant-inspired PCB design, I tested all connections and wiring with FireBeetle 2 ESP32-S3 and Beetle ESP32-C3. Then, I checked the BLE connection quality between Beetle ESP32-C3 and the Android application for transferring data packets.

Then, I designed my Iron Giant-inspired PCB by utilizing Autodesk Fusion 360 and KiCad. Since I wanted to design a unique 3D-printed PCB holder to simplify cable management, I designed the PCB outline on Fusion 360 and then imported it to KiCad. As mentioned earlier, I drew my inspiration from Iron Giant's industrial-esque vibe and captivating demeanor to create a unique mechanical anomaly detector.

To replicate this anomaly detector, you can download the Gerber file below and order my PCB design from PCBWay directly.

First of all, by utilizing a TS100 soldering iron, I attached headers (female), pushbuttons (6x6), a 5 mm common anode RGB LED, a long-shaft 10K potentiometer, and a power jack to the PCB.

📌 Component list on the PCB:

JF1, JF2 (Headers for FireBeetle 2 ESP32-S3)

JB1, JB2 (Headers for Beetle ESP32-C3)

S1 (Headers for Fermion IPS TFT LCD Display)

M1 (Headers for Fermion I2S MEMS Microphone)

RV1 (10K Long-shaft Potentiometer)

K1, K2 (6x6 Pushbutton)

D1 (5 mm Common Anode RGB LED)

P1 (Power Jack)
2
Step 1.1: Making connections and adjustments
```
// Connections
// FireBeetle 2 ESP32-S3 :
//                                Fermion 2.0" IPS TFT LCD Display (320x240)
// 3.3V    ------------------------ V
// 17/SCK  ------------------------ CK
// 15/MOSI ------------------------ SI
// 16/MISO ------------------------ SO
// 18/D6   ------------------------ CS
// 38/D3   ------------------------ RT
// 3/D2    ------------------------ DC
// 21/D13  ------------------------ BL
// 9/D7    ------------------------ SC
//                                Beetle ESP32 - C3
// RX (44) ------------------------ TX (21)
// TX (43) ------------------------ RX (20)
&
&
&
// Connections
// Beetle ESP32 - C3 :
//                                Fermion: I2S MEMS Microphone
// 3.3V    ------------------------ 3v3
// 0       ------------------------ WS
// GND     ------------------------ SEL
// 1       ------------------------ SCK
// 4       ------------------------ DO
//                                Long-Shaft Linear Potentiometer 
// 2       ------------------------ S
//                                Control Button (A)
// 8       ------------------------ +
//                                Control Button (B)
// 9       ------------------------ +
//                                5mm Common Anode RGB LED
// 5       ------------------------ R
// 6       ------------------------ G
// 7       ------------------------ B
//                                FireBeetle 2 ESP32-S3
// RX (20) ------------------------ TX (43) 
// TX (21) ------------------------ RX (44)
```
#️⃣ Before testing connections and wiring on a breadboard, I needed to solder male headers to some components by utilizing the soldering iron.

#️⃣ Since FireBeetle 2 ESP32-S3 provides a built-in camera interface with an independent camera power supply (AXP313A), I was able to attach the provided OV2640 camera effortlessly.

#️⃣ Since FireBeetle 2 ESP32-S3 and Beetle ESP32-C3 operate at 3.3V logic level voltage, I did not encounter any issues while connecting their hardware serial ports together.

#️⃣ Even though the Android application lets the user record audio samples via the phone microphone, since Beetle ESP32-C3 does not have an integrated microSD card module, I needed to connect a Fermion I2S MEMS microphone to Beetle ESP32-C3 in order to run the neural network model for audio classification in the field.

#️⃣ Then, I inserted a microSD card into the microSD card module on the Fermion TFT LCD display to save image samples easily. I also utilized the Fermion TFT LCD display to inform the user of the performed operations by showing the feature-associated icons.

#️⃣ Although the Android application allows the user to select a component (part) class to save image samples, I connected two control buttons and a long-shaft potentiometer to Beetle ESP32-C3 so as to provide the user with the option to save image samples and run the object detection model manually for debugging. Also, I added an RGB LED to inform the user of the device status while performing different operations.

#️⃣ After completing soldering and adjustments, I attached all remaining components to the Iron Giant PCB via the female headers.
3
Step 2: Designing and printing the Iron Giant-inspired case
Since I focused on building a budget-friendly and accessible mechanical anomaly detector that detects sound-based mechanical anomalies and diagnoses the root cause of the detected deviations via object detection so as to notify the user via SMS, I decided to design a robust and compact case allowing the user to place the Iron Giant PCB and position the OV2640 camera on FireBeetle 2 ESP32-S3 effortlessly. To avoid overexposure to dust and prevent loose wire connections, I added a removable top cover mountable to the main case via triangular snap-fit joints. Then, I designed a unique PCB holder by utilizing the PCB outline, mountable to the top cover diagonally via M3 screws and nuts. Also, I decided to emboss gear icons and the DFRobot symbol on the top cover to emphasize the capabilities of this AI-based mechanical anomaly detector.

Since I drew my inspiration from Iron Giant's industrial-esque vibe while designing this mechanical anomaly detector, I decided to print an Iron Giant bust to highlight the design similarities.

I designed the main case, the removable top cover, and the PCB holder in Autodesk Fusion 360. You can download their STL files below.

For the Iron Giant bust accentuating the device design, I utilized this model from Thingiverse:
- Iron Giant Head
Then, I sliced all 3D models (STL files) in Ultimaker Cura.

Since I wanted to create a metallic structure for the device case and apply a unique industrial-esque theme manifesting mechanical plants, I utilized this PLA filament:
- ePLA-Metal Antique Brass
Finally, I printed all parts (models) with my brand-new Anycubic Kobra 2 Max 3D Printer.

Since Anycubic Kobra 2 Max is budget-friendly and specifically designed for high-speed printing with a gigantic build volume, I highly recommend Anycubic Kobra 2 Max if you are a maker or hobbyist needing to print large prints without compartmentalizing your design and losing structural integrity while working on multiple prototypes before finalizing a complex project.

Thanks to its upgraded direct extruder and vibration compensation features, Anycubic Kobra 2 Max provides 300 mm/s recommended print speed (up to 500 mm/s) and enhanced layer quality. Also, it provides a high-speed optimized cooling system, reducing visible layer lines and complementing the fast printing experience. Since the Z-axis has dual-motors and dual-support rods, it prevents vibration from affecting layer smoothness and integrity, even at higher print speeds.

Furthermore, Anycubic Kobra 2 Max provides a magnetic suction platform on the heated bed for the scratch-resistant PEI spring steel build plate, allowing the user to remove prints without any struggle, even for larger prints up to 420x420x500 mm. Most importantly, you can level the bed automatically via its user-friendly LeviQ 2.0 automatic leveling system and custom Z-axis compensation. Also, it has a smart filament runout sensor and supports Anycubic APP for remote control and management.

#️⃣ First of all, remove all fixing plates. Then, install the gantry frame and support rods.

#️⃣ Install the print head and adjust the X-axis belt tensioner. Then, install the touchscreen and the filament runout sensor.

#️⃣ Connect the stepper, switch, screen, and print head cables. Then, attach the filament tube and use cable ties to secure the cables properly.

#️⃣ If the print head or bed is shaking, adjust the hexagonal isolation columns underneath them.

#️⃣ To avoid software-related print failures, update the device firmware manually via USB or directly over Wi-Fi.

I encountered some errors due to Cura configurations before the official 2.3.6 firmware.

#️⃣ After the firmware upgrade, go to Settings ➡ More Settings ➡ Guide so as to initiate the LeviQ 2.0 automatic bed leveling system and configure vibration calibration.

#️⃣ Finally, fix the filament tube with the cable clips, install the filament holder, and insert the filament into the extruder.

#️⃣ Since Anycubic Kobra 2 Max is not officially supported by Cura yet, we need to set it manually. Fortunately, Anycubic provides detailed configuration steps for Anycubic Kobra 2 Max on Cura.

#️⃣ First of all, create a custom printer profile on Cura for Anycubic Kobra 2 Max with given printer settings.

#️⃣ Then, import the printer profile (configuration) file provided by Anycubic, depending on the filament type.