Based on what's working, the ideal method would be absolute differencing every new box against the previous best box.  That would require a scale invariant method or the matches would be pretty bad.  The next best idea is histogram feature extraction.  All a histogram does is increase the number of target colors to compare.

That fixed 5 cases where the single color comparison failed, but added 1 new failure. 

Helas, success highly depends on the number of frames averaged.  15 frames fixes the bad case.  20 frames fails it because it weighed more data from outside the shade.  Histogram feature detection was better in every case, with the same averaging time.  It could be 1 second of averaging is the ideal amount, but it would have still failed if the lion entered the shade 5 frames later.  All the other cases weren't affected by the averaging time.  There's a compromise between responding quickly to lighting changes & washing out detections of the wrong animal.  There could be a rule where differences under .01 have to revert to largest box or get rejected.

-------------------------------------------------------------------------------------------------------------------------

Was surprised to discover the higher number of bins giving greater separation.  It was always assumed the chi square calculation wouldn't overlap higher numbers of bins but in reality, it seems to store more spatial information. 

Sadly, that mall scene with so many lighting changes was brutal.  256 bins fell over in some parts while improving others.  After much dialing of values, 32 bins & 15 frames got all the way through that 1 scene without any glitches.  There still is some potential for chi squared to misalign higher numbers of bins & some need for the moving average to keep up with lighting changes.  The best this algorithm can do is shoot for a middle ground.  The captured frame rate was 15 so it probably needs 12 frames of averaging during realtime tracking.

These are worst case scenarios.  The typical empty bike path hasn't had problems in dog years.

The hardest one on the bike path got lucky because the lion was in a more stable lighting while the wild animal was in & out of shadows.

Thus came a compilation video of color histogram feature extraction passing all the test cases.