I finished the enclosure by adding cutouts for a switch, the USB cable, and an extra reset/program button for the Teensy.

The perfboard is also finished for now, I'll only need to modify it again when I add the touch wheel. The buttons are now connected to a PCF8574A, because I might need I²C later anyway. Here's a picture without the Teensy and cables:

And one with the Teensy and the cable that goes to the main device:

The small switch in the lower right corner will later be used to enable power.

When the code really locks up, the Teensy downloader can't just reset the board over USB, so I added an extra reset/program button that can be accessed without opening the enclosure. It sits next to the user buttons and can be pressed through a small hole in the front which was not yet drilled in this picture:

A very simple piece of example code has shown that everything is hooked up correctly (I can read the buttons).