By playing with sample size and ADC resolution the phase output error is now under a degree.  Quite a win over all the time domain phase crossing detection hardware and filters.  I'm sure with some better audio signal isolation would help.

There is a new problem - the phase changes (~10 degrees) with input signal large amplitude changes.  This is very pronounced when the signal is very low or or on the verge of overload.  Need to look into setting the receiver volume to get the best results.

So far I also haven't done anything about the signal riding on ADC input - I haven't addressed the half scale DC offset built into the samples. 

Latest version of the code was  just posted.