I finally managed to hook everything up so that qmk, tinyusb and esp-idf all play nicely together, and managed to get that first glorious "a" key press, from the on-board button on the saola board.

There's still a lot of functionality that doesn't work yet, but I'm super happy that all the USB stuff is working well enough that it's able to enumerate and actually send an HID report on a button press.