That's a bit higher than I expected, but within reason. It seems like the timing numbers being collected have some validity.
So, first easy win is to switch some variables over to 8 bit integers. There's basically 2 pins that are being used to read values, so there's no reason to use anything larger than 8 bits.