So guys this is something I've been working on. For TTS on an MCU you usually end up with something like SAM, or you buy a more expensive part, or you give up and use an SBC. Neural TTS is making headway on MCU class devices but none of what I found met my goals of real time, lightweight and sounding decent. I wanted something more natural than SAM without needing high end hardware to run it.

So I ended up with this. It's a two stage neural pipeline feeding either a portable FARGAN reimplementation or a weightless LPC vocoder. The prosody model is a bidirectional GRU, 64 units, 8 dimensional word embeddings, 198,136 parameters, trained on LJSpeech. The acoustic model is a dilated convolutional network, 403,604 parameters across seven layers. All int8 quantized. Runs on a CH32H417 with no OS and no network.

First build was 85x slower than real time. Final one is 1.24x.

The change that mattered most was not arithmetic. The acoustic model was sweeping 397KB of weights out of flash once per frame because the frame loop sat inside each layer. Hoisting the tap loop and staging one 16KB tap into RAM per window took that stage from 8371 to 964 ms per second of audio, with bit identical output. That single change beat every arithmetic optimisation combined.

Using an int32 accumulator instead of float32 in an int8 kernel is a factor of 32 on a core without hardware float, and it's bit exact, so one code path serves both target classes.

Enabling the FPU delivered 4.3x where instruction counts predicted 32x. That gap is why I test on hardware instead of trusting simulation.

Next up is more testing and scoring, getting it onto other RISC-V parts, and closing that last bit to hit real time.