Close
0%
0%

Bee monitoring camtrap

To monitor bee behavior with videos, I designe a whole pipeline including micro I2S, mmwave radar, bayesian neural network

Similar projects worth following
0 followers
Bee monitoring with esp32p4 and mmwave radar

## 🐝 About DynamicBees
**DynamicBees** is an open-source, "FAIR-by-design" automated ethology and smart-garden monitoring kit designed to track insect behavior, dynamics, and environmental stressors. Unlike conventional systems that rely solely on species identification or heavy data compression, DynamicBees focuses on the **temporal dimension of behavior** and continuous micro-doppler/video recording.

To preserve absolute data integrity for complex ecological analysis, **our hardware performs no mathematical trajectory compression (such as TDA)**. Instead, raw video streams, audio data, and high-frequency radar metadata are safely stored directly onto high-capacity local storage (eMMC/SD) upon behavioral trigger confirmation.

---

## 🛠️ System Architecture & Hardware Stack
The project is built around the powerful **ESP32-P4** dual-core microcontroller, using a dual-path pipeline (Fast Path on Core 0 for real-time inference, Slow Path on Core 1 for bulk storage and logging):

1. **Wake-up Sentinel:** 24 GHz mmWave Radar (LD2410C) connected via GPIO 4 to wake the system upon movement.
2. **High-Precision Target Analysis:** 60 GHz mmWave Radar (SPI bus) for micro-doppler wingbeat frequency extraction and 3D spatial positioning ($z$-axis).
3. **Audio Capture:** I2S Digital Microphone for acoustic wingbeat frequency analysis (Goertzel & FFT algorithms).
4. **Vision & Edge AI:** H.264 Camera module managed via SCCB/I2C for lightweight FOMO/vision scoring.
5. **Environmental Context:** T/HR Sensor (I2C) tracking ambient temperature and relative humidity.
6. **Storage:** High-speed eMMC storage (SDMMC bus) preserving raw `.h264`, `.pcm` (audio), and `.meta` binary logs.
7. **Power Management:** Software-driven power-down (`PWDN`/`EN`) pins to completely shut down power-hungry sensors and enter deep sleep during inactive periods, maximizing energy efficiency.

---

## 🧠 Algorithmic Pipeline
- **Core 0 (Fast Path):** Continuously processes low-level sensory inputs (Audio Goertzel filters, 60GHz radar ramps, and lightweight vision inference). A **Bayesian Confirmation Network** evaluates the combined probabilities ($P(\text{Bee} \mid \text{Sensors})$) against a configurable threshold.
- **Core 1 (Slow Path):** Triggered only upon target confirmation. Manages circular buffers in PSRAM, flushes pre-trigger data, and streams synchronized multi-modal data streams directly to the eMMC storage files until target loss or session timeout.

  • 1 × ESP32-P4 (Dual-core RISC-V microcontroller with advanced vision and machine learning support)
  • 1 × Radar mmWave 24 GHz (LD2410C) (Low-power motion sentinel / wake-up trigger)
  • 1 × Radar mmWave 60 GHz SPI Module (High-resolution micro-doppler & range bin tracking)
  • 1 × I2S Digital Microphone (Acoustic wingbeat frequency monitoring)
  • 1 × H.264 Camera Module (Visual tracking & lightweight Edge AI vision)

View all 6 components

  • Hello World: Automated Ethology for Citizen Science

    Henri • an hour ago • 0 comments

    Welcome to the first log of the DynamicBees project! 🐝

    We started this project to address a critical gap in biodiversity monitoring. While insect populations are facing unprecedented challenges, we still lack scalable, accessible ways to monitor their behavior continuously in the wild. Traditional methods often stop at simple species identification or rely on heavy mathematical data compression that loses vital context.

    Our vision is to build an open-source, automated ethology tool designed specifically for citizen science. By combining the ESP32-P4, mmWave radar (60GHz for micro-doppler), and Edge AI, we can track the temporal dimension of insect behavior. More importantly, our "FAIR-by-design" approach ensures that raw, uncompressed data (audio, video, and radar metadata) is saved directly to local storage—meaning no mathematical compression (like TDA) altering the reality of the field.

    We need your help! DynamicBees is a collaborative citizen science initiative. To turn this prototype into a deployable "smart-garden" kit, we are actively looking for contributors and collaborators in⚡ Hardware / PCB Design: To help us miniaturize the current ESP32-P4 architecture into a compact, low-power, and robust board.

    If you are passionate about biodiversity, open-source hardware, or edge computing, we would love to have you on board. Drop a comment below, follow the project, or send me a message to start chatting. Let's build this together!

View project log

  • 1
    ### Step 1: Hardware Wiring & Pinout


    Connect your peripherals to the ESP32-P4 according to the official pin mapping defined in the firmware:
    - **24 GHz Radar IRQ (Sentinel):** Pin GPIO 4
    - **60 GHz Radar (SPI High-Speed):** CS = Pin 10, MOSI = Pin 11, SCK = Pin 12, MISO = Pin 13
    - **I2S Microphone (Audio):** BCLK = Pin 20, WS = Pin 21, DIN = Pin 22
    - **Camera H.264 (SCCB/I2C Control):** SDA = Pin 7, SCL = Pin 8
    - **Ambient T/HR Sensor (I2C):** SDA = Pin 2, SCL = Pin 3
    - **eMMC Storage (SDMMC Bus):** CLK = Pin 39, CMD = Pin 40, D0-D3 = Pins 41-44
    - **Power Down Control:** Radar 60GHz PWDN = Pin 5, Camera PWDN = Pin 14

  • 2
    ​ ### Step 2: Firmware Setup & Compilation


    1. Clone the repository or set up your ESP-IDF environment for the ESP32-P4.
    2. Ensure PSRAM is fully enabled in `menuconfig` (at least 2MB allocated for circular buffers as defined by `PSRAM_BUFFER_SIZE`).
    3. Flash the firmware (`edge.cpp` pipeline code) to your ESP32-P4 board.

  • 3
    Step 3

View all 3 instructions

Enjoy this project?

Share

Discussions

Does this project spark your interest?

Become a member to follow this project and never miss any updates