I figured out a way to preserve the 2 µs turn-off delay and still have a brighter display (comparable to what was there before).

The display raster timer will now alternate between two counts - a short value for the turn-off delay time, and a longer value for the actual display.

With this configuration, we can go back to a 10 kHz raster rate, but with a 75% duty cycle (actually, 75% of 12.5% since the display lights one digit at a time). This doesn't sound like much, but it's actually dramatic from what you can see.

The bad news is that the accuracy spec (as you'll recall, it's the PPS ISR latency plus the worst-case display refresh) now becomes 125 µs. But again - that's a worst case value. Since a display update resets the raster, we display the least significant digits immediately, so their latency will be much closer to 25 µs. It's only the hour digit an AM/PM indicators that wind up taking the worst-case time, and they don't change nearly as often.