I live in neighborhood with a lot of squirrels living in the old oak trees. I like the squirrels but I hate when they eat my heirloom tomatoes. I also don’t like the fact they eat all the nectarines off my tree before they are ripe. I don’t mind so much they eat the apples and pomegranates as we usually have enough to share.

I don’t want to kill the squirrels, but I do want to figure out a way to keep them out of my tomatoes. I’m thinking of building an automatic sentry gun that shoots airsoft bb pellets at the squirrels. An automatic sentry gun is a gun that is mounted so it can rotate in the horizontal (pan) and vertical (tilt) axis. Mine would be computer controlled using a microcontroller (arduino) and two or three servos.

There was a site https://github.com/sentryGun53/Project-Sentry-Gun that had sentry gun that fired on movement. I want to do one better. I want to only shoot at squirrels. Not people, birds, or cats. In fact I want to make the gun visibly safe when a person is in the area by lowering a solid piece of plastic in front of the muzzle when it detects a person.

Phase 1 of this project is to investigate how to detect squirrels with machine learning. Phase 2 is to build a sentry gun that only targets squirrels (Squirrel Season!). There might be a Phase 3 where I mount a camera and an airsoft gun on a drone (Skynet for Squirrels?)

Phase 1 is essentially complete. Phase 2 hasn’t started yet. Here’s what I’ve learned from Phase 1.

Update 6/4/2018: I'm revamping what I did with Phase 1 to use a Jevois camera system. http://jevois.org/ The cam runs linux and has Darknet/Yolo preloaded. My thought is I can use it plus the RaspberryPi and won't need an additional laptop. I have written a python script that can get the results of the detection (what type and the coords) from the Jevois camera on the Pi. That all works great with the standard Yolo network it ships with.

Next, I put on the custom network that I made as part of Phase 1 and run into some roadblocks. The current version has YoloV2 so I have been retraining my network to use a Yolov2 compatible network. I'm in the middle of figuring out why that doesn't work right. When I have that sorted, I'll post the code and do a proper writeup.

Update: 6/30/2018: I'm still working to figure out how  to get the cam to reliably detect the objects I want. I've trained a network based on Yolov2-tiny-voc. It reliably detects people and squirrels on my laptop but not on the Jevois. I suspect it's because the systems are running different versions of Yolo but I'm not sure if it's something else.  I'll pretty happy with my python code that runs on either my windows PC or my Raspberry PI 2B and runs the cam. I'll post that at the bottom of this writeup.

Update 7/31/2018: I have the camera working with Yolov3 with the python code running on a Raspberry Pi 3. I've done multiple attempts at training the network but I have not succeed in detecting a squirrel in a live feed. I saw one sitting in the camera range yesterday and it didn't recognize it. I've tried turning down the threshold until it starts recognizing too many bogus things. I have one more thing to try (upping the network resolution per https://github.com/AlexeyAB/darknet/blob/master/README.md#how-to-improve-object-detection) and if that doesn't work, I'm going to capture a boatload of  pictures off the camera directly and train from that. I've written a python script to do it and we'll see how that goes. By the way, I've started posting the code on my Github site: https://github.com/PeterQuinn925

Update 9/5/2018: Success detecting squirrels! I'm not quite getting it live, but I am getting it through the Jevois camera. I have it taking shots when there's movement. With the background being trees, it just takes a tiny bit of wind to trigger a shot. I've been paging through thousands of...