Well I took another look at the MPLAB Code Configurator for the firmware and I realized I was only running the CPU core at something like 8MHz.  So I modified the multiplier so it's running at 48MHz and the improvement was significant.  Flash program and verify now takes 35 seconds and @Xark reports SRAM programming takes only 2 seconds.  Very nice!  This is before any bulk serial optimizations have been added.  I'm very happy with this result.

Next steps: I want to add the bulk serial optimizations in and see even better performance.  Then I will add support for programming .jed and .bit files directly, and finally I will add the module into the TinyFPGA Programmer GUI that exists for #TinyFPGA B-Series.

Once that is done and working to my satisfaction I'll be making some more revisions to the PCB.  I think it makes sense to breakout all the PIC's pins so that the board can be used for other purposes as well.  So I can add support for UART and SPI along with it's existing GPIO capabilities. That will make it a dirt cheap programmer that can be used for many things.