Component selection should be final now, the power supply part was missing.
The battery charger, 3.3V step-down, and 5V step-up for the audio amp are chosen.

Regarding the CPLD itself, I realized earlier today that the GB APU would never fit in 160 blocks. So I went towards FPGAs. Expensive FPGAs. Which need 3 different power supplies, flash,...

I was ready to go for a $25 Spartan3A and say goodbye to board space and battery life when @Gozilu reminded me of the new MAX 10 series from Altera !

I usually refrain from using new products because of fear or complexity, but those seem perfect for this application: 2K LE CPLD with 13kB of BRAM and sleep mode for $8 ? YES PLEASE !

Here's the board outline with all the main components footprints.
The CPLD is a 0.8mm pitch BGA, less scary than the BGA36 version of it, at 0.5mm. A waste of I/Os but it doesn't matter.

CPU and memory are on the left, power supply and audio on the right. Display pads on the back (blue).