Ai Equiped Wasp (and Asian Hornet) Sentry Gun

• Lasers are dangerous when used out in the open.
• Gimbal is not accurate enough.
• Camera will probably never pick up wasps/hornets in flight.
• Variable outside lighting conditions will always be a problem.

The proposed solution is to police the entrance to the hive by forcing the creatures to travel up the length of a 25 mm aluminum box section which has a camera mounted in the middle, looking inwards. I checked the Logitech C230e focus and it is effective to 22mm from the front of the camera, so 25 mm box is perfect. There's also a 1000 mA laser head available with 25 x 25 mm front profile. The insects will crawl over the top of the laser and get spotted by the camera as they go. If a wasp or hornet is spotted ..... they get blasted by the laser. The bees *should* clear out the dead bodies themselves if they are not completely vapourised for some reason.

Small LEDs can be mounted at strategic locations within the box tunnel to help the camera.Currently they are green, as those were the only ones i had lying around.

Here's a camera side view:

This is with only a couple of LEDs. Once the lighting is sorted, the image will be very nice - perfect for feeding into the object detection system.

As mentioned in previous logs, there was something seriously wrong with the model trained using mobilenet SSD …… And I don't know what went wrong …. The training and subsequent model optimisation seemed to run smoothly. I ran out of time and patience so have gone back to the Jetson TX2 and made a 'like for like' appraisal using a 'fresh' video from YouTube for testing the detection capabilities. It's important to test the network on footage that was NOT used to train the network.

Using an external USB camera, I could see no false positives on the TX2 and it did not pick up on similar insects such as flies. It's also worth noting that the network was trained using quite a few images with honey bees in the back ground, which will help focus the results away from similar looking insects. 

The bvlc_googlenet model was trained on Amazon Web Services (AWS) for about 5 hours, 63300 iterations with a approx. 2,500 images set, 640 x 640 pixels.

The servos used here are a bit 'rough' in operation and there are much smoother servos available, particularly the Hitec 8XXX and 9XXX series which claim 12 bit (4096) operation with deadbands of 0.008 us. They are, however, rather pricey:

https://www.multiplex-rc.de/produkte/detail/index/sArticle/6645

Training is going well with loss stats gradually tapering off as the model becomes more accurate. Another 2 days to go.

Since the results of the blob inference time documented in the previous log was a bit disappointing, I thought I'd try retraining the MobileSSD network with no 'pretrained model' as i thought that this model was probably what prevented me reducing the number of classes to train option. All the solverstate files and caffemodels had to be deleted in the 'results' folder as the software tried to carry on from the previous train. Initially the software complained that their was no pretrained model, but setting: 'pretrain_model = ""' fooled the python script into carrying on regardless. 'num_classes must be set to a minimum of 2 as their always needs to be a 'dont care' class.

The model is now being trained on the Jetson TX2, which is a bit slow for this kind of work, but it's all I've got other than messing about on AWS. It'll take at least 2 days, maybe a lot longer .... Time will tell!

After abandoning attempts to use C++ as shown in previous log, I reverted to python scripts, which are a lot easier to 'compile'. Hackaday's own Lewin Day pointed towards Les Wilkinson's work with Python and Neural Network Stick 2 and Intel Openvino and after forking Les's repository ( https://github.com/paddygoat/RPi3_NCS2 ) I was quickly able to modify his bottle following robot code to make my gimbal track not only bottles, but also faces and wasps. Having already proven my wasp model to work on openvino, I  now added timers to the python script to track down bottlenecks and found that my model had a fairly large one around the 'inference blob' in the script, which was, to me at least, really interesting!

Using the same script, I was able to demonstrate the difference in performance between two networks, namely 'MobileNet-SSD' and 'Squeezenet 1.0 modified SSD', finding that the older Squeezenet model ran a lot more efficiently on the Raspberry Pi configuration giving higher blob rates and very much better inference accuracy / precision. If the object of interest is relative simple, like a face, it might be worth looking more closely at an older and simpler network such as Squeezenet for getting better results than more complicated networks such as MobileNet SSD.

This is the rough directory structure of the two programs I am trying to combine on Raspberry Pi. There's different kinds of make files all over the place, many duplicating each other, many saying 'do not edit'. It's a tricky puzzle and I have very little idea what I am doing!

Basically it's Intel OpenVino + servo motor pca9685 libraries on Raspberry Pi. Both libraries work fine independently, but wont compile together due to missing dependencies linked to pca9685. If anybody can help, please get in touch!

To keep things nice and easy, I used the Adafruit 12 bit pwm servo hat: https://learn.adafruit.com/adafruit-16-channel-pwm-servo-hat-for-raspberry-pi?view=all and a standard library written in C++ for coding. There are python libraries, but I'm going to need to combine the Gimbal code with another C++ program,  object_detection_demo_ssd_async/main.cpp, which is used for the inference procedure.

Getting the gimbal moving at alarmingly crazy and jerky speeds was fairly easy, but to get it running nice and slow and smooth took a little bit more head scratching. I changed the lib-pca9685 https://github.com/paddygoat/rpidmx512/tree/master/lib-pca9685 simple.cpp file to get much better resolution than with the 'proper' servo.cpp one.

First of all, install the dependencies and libraries:

wget http://www.airspayce.com/mikem/bcm2835/bcm2835-1.58.tar.gz
tar zxvf bcm2835-1.58.tar.gz
cd bcm2835-1.58
./configure
make
sudo make check
sudo make install

cd /home/pi/rpidmx512-master/lib-pca9685
make -f Makefile.Linux "DEF=-DRASPPI"

Compile and build the examples on Linux Raspbian:

cd /home/pi/rpidmx512-master/lib-pca9685/examples
make

 Next, run the 'servo.cpp' or 'simple.cpp' file:

cd /home/pi/rpidmx512-master/lib-pca9685/examples && make && sudo ./servo
cd /home/pi/rpidmx512-master/lib-pca9685/examples && make && sudo ./simple

/**
 * @file simple.cpp
 *
 */
/* Copyright (C) 2017-2018 by Arjan van Vught mailto:info@raspberrypi-dmx.nl
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:

 * The above copyright notice and this permission notice shall be included in
 * all copies or substantial portions of the Software.

 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
 * THE SOFTWARE.
 */

#include <stdio.h>
#include <stdint.h>
#include <unistd.h>
#include <iostream> 

#include "pca9685.h"
using namespace std; 
int msleep(unsigned long milisec)   
{   
    struct timespec req={0};   
    time_t sec=(int)(milisec/1000);   
    milisec=milisec-(sec*1000);   
    req.tv_sec=sec;   
    req.tv_nsec=milisec*1000000L;   
    while(nanosleep(&req,&req)==-1)   
        continue;   
    return 1;   
}   

int main(int argc, char **argv) {
    if (getuid() != 0) 
    {
        fprintf(stderr, "Program is not started as \'root\' (sudo)\n");
        return -1;
    }

    uint16_t OnValue, OffValue;
    PCA9685 pca9685;

    pca9685.Dump();
    pca9685.SetFrequency(100);

    //pca9685.SetFullOn(CHANNEL(0), true);                     // Channel 0 Full On
    //pca9685.Write(CHANNEL(1), VALUE(PCA9685_VALUE_MAX/2));    // Channel 1, Duty Cycle = 50 %
    
    int a = 450;                                            // Laser points upwards.
    int b = 900;   // b must be greater than a.
    int c = 5;
    int x = 500;
    
    pca9685.Write(CHANNEL(0), VALUE(a));                     // Channel 2, Duty Cycle = 20 %
                                                            // Max value = 4096. 20 % = 819.2 ~ 819 counts
    //pca9685.SetFullOff(CHANNEL(3), true);                     // Channel 0 Full Off
    
for(int d=c; d>0; d--)
  {
    for(x =b; x>a; x--)
    {
      pca9685.Write(CHANNEL(0), VALUE(x));                  // Up and down
      pca9685.Write(CHANNEL(1), VALUE(x));
      printf("Value d: %i", d);
          printf(" Value x: %i", x);
          printf("\n");
      msleep(50);
    }    
    for(x =a; x<b; x++)
    {
      pca9685.Write(CHANNEL(0), VALUE(x));
      pca9685.Write(CHANNEL(1), VALUE(x));
      printf("Value d: %i", d);
          printf(" Value x: %i", x);
          printf("\n");
      msleep(50);
    }
  }    

    pca9685.Dump();

    printf("\nOutput (%dHz):\n", (int) pca9685.GetFrequency());

    pca9685.Read(CHANNEL(0), &OnValue, &OffValue);
    printf("\tChannel 0: %04xh-%04xh, Full On\n", OnValue, OffValue);

    pca9685.Read(CHANNEL(1), &OnValue, &OffValue);
    printf("\tChannel 1: %04xh-%04xh, Duty Cycle = 50%%\n", OnValue, OffValue);

    pca9685.Read(CHANNEL(2), &OnValue, &OffValue);
    printf("\tChannel 2: %04xh-%04xh, Duty Cycle = 20%%\n", OnValue, OffValue);

    pca9685.Read(CHANNEL(3), &OnValue, &OffValue);
    printf("\tChannel 3: %04xh-%04xh, Full Off\n", OnValue, OffValue);

    return 0;
}

 It's all working nicely. Video to follow soon!