I didn't find this bug by looking for it. I found it while bench-testing a new node: a XIAO ESP32-S3 moisture sensor with a Wio-SX1262 radio that reports its battery voltage in every heartbeat. I wanted to watch 'bat_mv' climb while the LiPo charged, so I kept resetting the sensor and watching Node-RED for each heartbeat.
They took minutes to show up, and then several arrived at once.
Following the packet
The path is sensor → LoRa Pico (RFM95, 915 MHz) → UART at 230400 baud with A5 5A framing → Wiznet Pico → websocket → Node-RED → MQTT. With both the LoRa Pico and the Wiznet board plugged into the same machine, I logged their serial output side by side.
The LoRa Pico was fine. It received each packet, sent the ACK, and handed the frame to the Wiznet within the same second. One packet left the LoRa Pico at 5:17:01 and reached Node-RED at 5:19:17. Two heartbeats from separate boots, 107 seconds apart, arrived at the same instant, and one burst had packet IDs 244, 245, 246 and 245 again. Obviously, that wasn't correct.
First theory: packet size
The sensor's early packets were chatty JSON, about 340 bytes at the bridge once base64'd and wrapped. The bridge has historically struggled with big frames, so I trimmed the payload down to the essentials (at most 95 bytes over the radio, about 210 at the bridge) and moved the location and topic mapping into Node-RED, keyed by node number (filtered in NodeRed by header_from = #)
It didn't help. The smaller frames got stuck too.
The numbers that gave it away
The Wiznet's status heartbeat carries counters, and over one minute they told the whole story:
- 'rxB' (bytes received) rose by exactly the size of the six frames the LoRa Pico sent.
- 'frames' stayed frozen at 4971.
- 'buf' (unparsed bytes) climbed 6787 → 7246 → 8162.
- 'bad' and 'parse_err' didn't move, so nothing was being rejected.
- 'sent_age' crept up to 95 seconds.
So the bytes were arriving, and the parser just wasn't getting through them.
Root cause
About 8% of UART frames arrive damaged, most likely overruns while the Wiznet is busy on the network. When a damaged payload hit _decode_payload, payload.decode('utf-8') threw a UnicodeError. The fallback, decode('utf-8', 'ignore'), doesn't help, because this MicroPython version ignores the errors argument and throws again. The exception was swallowed, the read position never moved, and every frame behind the bad one waited until the 8 KB buffer limit finally cut it off. Then the whole backlog went out in one burst.
A quick simulation of the parser reproduced it: bursts of up to 35 frames with the old code, and every frame delivered immediately with the fix.
The fix
- _decode_payload returns None on a bad payload instead of clogging the buffer.
- If the next frame's A5 5A header shows up inside the current frame's length, the current frame is garbled, so drop it immediately and resync at that header.
- After any skip, the parse loop keeps draining instead of waiting for the next read.
- Any other parse exception drops one byte and resyncs, and is counted in a new pexc field in the heartbeat.
A damaged frame now costs one packet instead of stalling everything behind it.
Result
After flashing, the Wiznet heartbeat showed frames: 19, sent: 19, buf: 0, bad: 0, pexc: 0. A sensor packet ACKed at 7:36:16 was on the websocket before the 7:36:19 heartbeat. No more batching.
And the battery test that started all this is now running for real.
Link to the fix
https://github.com/scottlecomte/Lora-to-Ethernet-Bridge
Moisture sensor project to be created soon!
Scott LeComte
Discussions
Become a Hackaday.io Member
Create an account to leave a comment. Already have an account? Log In.