After a couple more flights it became apparent that there is an issue with the ultrasonic sensors.

They work fine on the ground but in the air with the propellers spinning the have a hard time detecting anything. This is likely caused by the noise and air pressure created by the props. when flying. After a little research I found this paper

http://www.daedalus.ei.tum.de/attachments/article/66/Ultrasonic Sensor Operation on a Quadcopter.pdf

which reinforces my theory.

So a little disappointed I bought a sharp IR distance sensor on ebay hoping it won't have the same issues.

I also just got the rf24 receiver to send back data from the sensors now so tomorrow I can see more precisely where and when the issues start to appear. I've been wanting to add a battery monitor for the quad as well. It would be great to see the battery status on the controller so you know exactly how much voltage you have left without having to rely on these annoying battery beepers.