The "wiring" PCBs arrived, so I added the required headers to attach the ESP-32 to the rest of the components.

I soldered in sockets for the ESP-32 and I2S board. 

Thankfully assembly went well. SparkFun Electronics has some nice "Jumper Cables" in short lengths (4", 6", and 12") with many pin number options. They worked out great in this case. I designed and printed some clips to hold the wires flat as there is not that much clearance to the bottom of the case. 

The PCBs and TFT are held in place by L shaped tabs,  and the 7-Segment display is  screwed on so that it would be easy to print an new case and swap the electronics into it.

I still have to add an external power connector and front panel switch (I'm temporarily running in a USB cable to power the Trekulator and  so I can program the ESP-32), but the hardware is basically done. 

Time to write a calculator program.