Digital twin-enabled Smart Shipping Workstation

Details

At the pinnacle of industrial artificial intelligence and machine learning applications, a digital twin represents a virtual construction of real-world physical products, mechanisms, or mechanical procedures. Since the simulation of a real-world product or industrial technique provides the flexibility of exerting countless examination scenarios, even to the point of cycling arduous or dangerous tasks, without causing any tangible ramifications and safety risks, digital twins hold immeasurable importance in developing adaptive product manufacturing procedures and building secure, cost-effective, and efficient industrial facilities.

After inspecting recent research papers about the applications of digital twins in industrial operations, I noticed the focal point of employing a virtual representation is to improve the safety and efficiency of an already existing industrial facility or mechanical process. Even though forestalling acute physical hazards due to dangerous work safety risks and advancing the precision of ongoing industrial operations are the prominent digital twin use cases, I wanted to explore the innovative opportunities of reversing the digital twin implementation and starting with a virtual industrial construction, consisting of individual machinery and sample product components, to develop a safe, practical, cost-effective, and efficient real-world mechanism from scratch.

By reversing the digital twin application process, I wanted to investigate whether having a virtual construction before building the real-world counterpart could help to forfend concomitant risks of assembling an industrial manufacturing system, reduce exorbitant overhaul costs due to the lack of internal design blueprints, and test device components to obtain the optimum performance for multifaceted operations.

As I was conceptualizing this proof-of-concept project, I inspected various industrial settings with which I could show the benefits of reversing the digital twin implementation. Since product transportation and shipping operations require complex industrial mechanisms to achieve accuracy and reliability while maintaining a time-sensitive workflow, I decided to apply my reverse digital twin approach to design a virtual shipping workstation, construct a synthetic data set of customized sample products, and train a precise object detection model to accomplish building a production-ready product transportation mechanism. In accordance with my approach, I designed all sample products from scratch to emphasize the strengths of a full-fledged digital twin, providing the opportunity to train an object detection model for products waiting to be manufactured.

Since I needed to know the exact electronic components employed by the shipping workstation to create compatible 3D parts, I decided to prototype the mechanical device structure and design a unique PCB (inspired by Wall-E) based on Arduino Nano Matter as the workstation control panel. I designed the Wall-E PCB outline and encasement on Autodesk Fusion 360 to place the electronic components in relation to the PCB easily while designing the virtual shipping workstation.

After testing electronic components and completing the PCB layout, I designed a plethora of 3D parts on Autodesk Fusion 360, including but not limited to custom bearings optimized for 5 mm steel balls, planetary gear mechanisms, and separated rotating platforms. After finalizing the required mechanical 3D parts, I exported the virtual shipping workstation as a single file in the OBJ format to produce an accurate virtual representation of the shipping workstation.

Then, I imported the virtual shipping workstation into the NVIDIA Omniverse USD Composer, which allows users to assemble, light, simulate, and render large-scale scenes for world-building. To generate a realistic scenery for shipping operations, I utilized some free 3D models provided by Omniverse and designed some additional assets. After completing my shipping...

Components

1 × ELECROW Custom PCB

1 × Raspberry Pi 5

1 × Arduino Nano Matter

1 × USB Webcam (PK-910H)

4 × Nema 17 (17HS3401) Stepper Motor

Build Instructions

Collapse

1

Design process, available features, and final results

As my projects became more intricate due to complex part designs, multiple development board integrations, and various features requiring interconnected networking, I decided to prepare more straightforward written tutorials with brevity and produce more comprehensive demonstration videos showcasing my entire design process, results, and device features from start to finish.

Thus, I highly recommend watching the project demonstration videos below to inspect my design process, the construction of the synthetic data set, and all of the shipping workstation features.
2

Step 0: A simplified illustration of interconnected networking

As a part of preparing a more visually-inclined tutorial, I decided to create a concise illustration of interconnected networking infrastructure to delineate the complicated data transfer procedures between different development boards, complementary web, and mobile applications.
3
Step 1: Testing electronic components and prototyping the device structure
Before proceeding with designing 3D parts, I needed to determine all electrical components required to operate the real-world shipping workstation. Thus, I started to test and prepare electronic components for prototyping the device structure.

#️⃣ Since Arduino Nano Matter is a versatile IoT development board providing state-of-the-art Matter® and Bluetooth® Low Energy (BLE) connectivity thanks to the MGM240SD22VNA wireless module from Silicon Labs, I decided to base the shipping workstation control panel on Nano Matter.

#️⃣ Since I envisioned a fully automated homing sequence for the moving workstation parts, I decided to utilize an IR break-beam sensor (300 mm) and two micro switches (KW10-Z5P).

#️⃣ By utilizing a soldering station for tricky wire connections, I prepared the mentioned components for prototyping.

#️⃣ Since I needed to supply a lot of current-demanding electronic components with different operating voltages, I decided to convert my old ATX power supply unit (PSU) to a simple bench power supply by utilizing an ATX adapter board (XH-M229) providing stable 3.3V, 5V, and 12V. For each power output of the adapter board, I soldered wires via the soldering station to attach a DC-barrel-to-wire-jack (male) in order to create a production-ready bench power supply.

#️⃣ Since Nano Matter operates at 3.3V and the IR break-beam sensor requires 5V logic level voltage to generate powerful enough signals for motion detection, the sensor cannot be connected directly to Nano Matter. Therefore, I utilized a bi-directional logic level converter to shift the voltage for the connections between the IR sensor and Nano Matter.

#️⃣ Since I planned to design intricate gear mechanisms to control the moving parts of the real-world shipping workstation, I decided to utilize four efficient and powerful Nema 17 (17HS3401) stepper motors, similar to most FDM 3D printers. To connect the Nema 17 stepper motors to Nano Matter securely, I employed four A4988 driver modules.

#️⃣ As a practical shipping workstation feature, I decided to connect a tiny (embedded) thermal printer to Nano Matter to print a shipping receipt for each completed order. I utilized a sticker paper roll to make receipts fastenable to cardboard boxes.

#️⃣ To build a feature-packed and interactive workstation control panel, I also connected an SSD1306 OLED display, three control buttons, and an RGB LED to Nano Matter.

#️⃣ As depicted below, I made all component connections according to available and compatible Arduino Nano Matter pins.
```
// Connections
// Arduino Nano Matter :
//                                Nema 17 (17HS3401) Stepper Motor w/ A4988 Driver Module [Motor 1]
// 3.3V    ------------------------ VDD
// GND     ------------------------ GND
// D2      ------------------------ DIR
// D3      ------------------------ STEP
//                                Nema 17 (17HS3401) Stepper Motor w/ A4988 Driver Module [Motor 2]
// 3.3V    ------------------------ VDD
// GND     ------------------------ GND
// D4      ------------------------ DIR
// D5      ------------------------ STEP
//                                Nema 17 (17HS3401) Stepper Motor w/ A4988 Driver Module [Motor 3]
// 3.3V    ------------------------ VDD
// GND     ------------------------ GND
// D6      ------------------------ DIR
// D7      ------------------------ STEP
//                                Nema 17 (17HS3401) Stepper Motor w/ A4988 Driver Module [Motor 4]
// 3.3V    ------------------------ VDD
// GND     ------------------------ GND
// D8      ------------------------ DIR
// D9      ------------------------ STEP
//                                Tiny (Embedded) Thermal Printer
// D0/TX1  ------------------------ RX
// D1/RX1  ------------------------ TX
// GND     ------------------------ GND
//                                SSD1306 OLED Display (128x64)
// A4/SDA  ------------------------ SDA
// A5/SCL  ------------------------ SCL
//                                Infrared (IR) Break-beam Sensor [Receiver]
// A6      ------------------------ Signal
//                                Control Button (A)
// A0      ------------------------ +
//                                Control Button (B)
// A1      ------------------------ +
//                                Control Button (C)
// A2      ------------------------ +
//                                Micro Switch with Pulley [First]
// A3      ------------------------ +
//                                Micro Switch with Pulley [Second]
// A7      ------------------------ +
//                                5mm Common Anode RGB LED
// D10     ------------------------ R
// D11     ------------------------ G
// D12     ------------------------ B
```
#️⃣ Furthermore, I put Raspberry Pi 5 into its aluminum case providing a cooling fan to secure all cable connections.