OMG, sounds just like my story, YouTube watching kids, etc. My CPU/GPU project [https://hackaday.io/project/180724-project-ember] is aimed at FPGAs, but similar always wanted to do it, just decided to go for it. Would be great to get an update if you're still working on this project!

Impressive!

If you're looking for an up/down counter faster than the 74HC191, you might consider the 74AC193 , 74F193, 74AC191, 74F191, all of which seem to be in stock at Digikey.com , and some of them seem to be in stock at Jameco.com . My understanding is that if you cascade 4 of the 74HC191 (to make a 16-bit stack pointer), the datasheet for suggests feeding /RCO output to the /CTEN input of the next one for (faster) synchronous counting, although it also mentions an alternative feeding /RCO output to CLK input for (slower) ripple counting.

Hi Louis, I suppose your CPU will be microprogrammed. You could use up-counters (HC161) for the SP and do the decrement throught the ALU in one or more extra microcode cycle(s). 

Perhaps my 16 bit #Kobold K2 - RISC TTL Computer, although very different, will also give you inspiration.

But I do not really understand your problem with the HC191 and the HC series. When I read the datasheet, it seems counting is sychronous so all your 4 counters will count at the same time. Max count frequency is 14MHz worst case, 42MHz typical. But the async load might indeed be a problem.

And the HC series seems to me the easiest to use of all logic families, and for more speed you can go to the AC series.

Wow somehow I hadn't seen that one yet in my searching! Very impressive I'm going to have a look through it.

I'm really new to this so take anything I say with a huge grain of salt. I thought it did up/down counting through ripple carry.  The way I have it wired right now the first nibble has CE pulled down by a NOR gate from the inc/dec control lines.  The RC output from the first nibble feeds the CE of the second and so forth for the third and fourth.  I was under the impression that the second nibble couldn't start it's count cycle until the first nibble passed it something on it's CE pin.  Thus the total time to count worst case would be the propagation times from CE to RC * 3 + CE to Q. I worked that out to 71ns.  I could be totally wrong. 

I'm planning my GPU to be entirely AC series because I'm going to run it at 25Mhz but they don't make an UP/DOWN counter in AC. 

As for sending the DEC through the ALU that was my first attempt.  Yes it's microcoded. No cool RISC for me yet. ( Next one is RISC V ! )  I have only 3 bits set for the stage counter ( due to inputs on the control roms ) so I have a maximum of 8 steps for any instruction.  The Jump Subroutine ( JSR ) instruction is already at 8 without going through the ALU.   Since JSR absolute is 2 DEC operations that's 10+ steps if using the ALU.   Soo for now I'm going to live with the HC191. The Async load I have worked out in the simulator and it only cost me 1 cycle on 1 instruction thus far extra.