I have published the code at https://github.com/m-bw/EyeBREAK; it includes the training notebook as well as the firmware.

I was testing if a 36x36 image input would work, and it in fact does achieve the same or better accuracy (97%) with a third of the parameters (10k) and also halves the inference time (20ms). This brings the FPS back up to around 28.

I also moved to esp-nimble-cpp for the BLE stack and it does indeed reduce flash usage by about .2MB.