Close

Facenet with fisheye cam on rasp 5

A project log for Raspberry pi tracking cam

Tracking animals on lower speed boards to replace jetson & coral

lion-mclionheadlion mclionhead 05/01/2024 at 06:570 Comments

At least 1 attempt at this needs to happen.  Facenet needs the highest resolution it can get, which requires defishing at least 810x540.  That's the limit of the useful resolution from the elp fisheye.  Then there would be a 2nd step to downscale it to efficientlion size.

The last of the code which used  movenet, the HDMI cam & the keychain cam was moved to truckflow.*.posenet

The higher resolution for facenet really needs smooth defishing.  The defish smoothing only works when it's not scaling.  It should probably defish at 1620x1080.

Opencv dropped support for the original yunet.onnx after 2022.  There's now a 16 bit face_detection_yunet_2023mar.onnx you have to track down. An 8 bit derivative doesn't work.

opencv_zoo/models/face_detection_yunet/face_detection_yunet_2023mar.onnx at main · opencv/opencv_zoo

AT 810x540, it goes at 5fps with efficientlion-lite1 going.  It drops to 2fps at 1620x1080.  810x540 goes at 8fps alone. Efficientlion-lite0 might be worth it.  The age old problem still exists where you have to look at the camera head on.  The fisheye lens doesn't do much for it. 

If it scans just the hit boxes from efficientlion, it goes at 8fps with 1 hit box.  It depends on the size & number of hit boxes.  The worst case would be  slower than just scanning the entire frame.

It would have to scrap tracking the largest box & use the nearest box to the last box with the matching face instead.  It also needs to dedup the hit boxes.  The problem with deduping is 1 hit box might have a face while another might not.  It's no guarantee.  What's really needed is a way to play recorded footage through the algorithm in place of the live camera & record the output without the phone.

Rough go of it with the training footage.  The face detector didn't hit anything.  It definitely works but the truck is too low.  Efficientlion-lite1 hit a lot of trees but might have just been common randomness.  It was about as bad as tensorrt. 

The efficietlion-lite0 model from 2022 had fewer false hits but also failed to detect anything in a lot more frames.  It might be worth running lite0 in FP16 & lower resolution.

Discussions