To detect x,y coordinates initially I used Haar cascades in RaspberryPI. After that yolov4-tiny in Jetson nano. For Y coordinates - stereo vision in OpenCV.
Calculation of necessary value for the angle of mirrors.
RaspberryPI/JetsonNano by SPI sends a command for galvanometer via DAC mcp4922. Electrical scheme (here). From mcp4922 bipolar analog signal go-to amplifier. Finally, we have -12 and + 12 V for the control positions of the mirrors.
The principle of operation
Single board computer to processes the digital signal from the camera and determines the positioning of the object, and transmits the digital signal to the analog display - 3, where digital-to-analog converts the signal to the range of 0-5V. Using a board with an operational amplifier, we get a bipolar voltage, from which the boards with the motor driver for the galvanometer are powered - 4, from where the signal goes to galvanometers -7. The galvanometer uses mirrors to change the direction of the laser - 6. The system is powered by the power supply - 5. Cameras 2 determine the distance to the object. The camera detects mosquitoes and transmits data to the galvanometer, which sets the mirrors in the correct position, and then the laser turns on.
Don't use the power laser!
The main limiting factor in the development of this technology is the danger of the laser may damage the eyes. The laser can enter a blood vessel and clog it, it can get into a blind spot where nerves from all over the eye go to the brain, you can burn out a line of "pixels" And then the damaged retina can begin to flake off, and this is the path to complete and irreversible loss of vision. This is dangerous because a person may not notice at the beginning of damage from a laser hit: there are no pain receptors there, the brain completes objects in damaged areas (remapping of dead pixels), and only when the damaged area becomes large enough person starts to notice that some objects not visible.
We can develop additional security systems, such as human detection, audio sensors, etc. But in any case, we are not able to make the installation 100% safe, since even a laser can be reflected and damage the eye of a person who is not in the field of view of the device and at a distant distance. Therefore, this technology should not be used at home.
My strong recommendation - don't use the power laser! I recommend making a device that will track an object using a safe laser pointer.
Dimensions
1 - PI cameras, 2 - galvanometer, 3 - Jetson nano, 4 - adjusting the position to the object, 5 - laser device, 6 - power supply, 7 - galvanometer driver boards, 8 - analog conversion boards
Galvanometer setting
In practice, the maximum deflection angle of the mirrors is set at the factory, but before use, it is necessary to check, for example, according to the documentation, our galvanometer had a step width of 30, but as it turned out we have only 2
Maximum and minimum positions of galvanometer mirrors:
a - lower position - 350 for x mirror;
b - upper position - 550 for x mirror;
c - lower position - 00 for y mirror;
d - upper position - 250 for y mirror;
Determining the coordinates of an object
X,Y - coordinate
Z-coordinate
We created GUI, source here
At the expense of computer vision, the position of the object in the X, Y plane is determined - based on which its ROI area is taken. Then I use stereo vision to compile a depth map and for a given ROI with the NumPy library tool - np.average we calculated the average value for the pixels of this area, which will allow us to calculate the distance to the object.
You can find more detail in the published paper in preprint - Low-Cost Stereovision System (Disparity Map) For Few Dollars
Determining the angle of the galvanometer mirror
angle of galvanometer...
Pondering a bit on the safety of people nearby this device; Perhaps a (circular) microphone array together with camera could function as a form of safety? If the camera detects any large movement in its field of view it disables the diode, if it doesn't AND the array detects mosquito wings (quite distinct, with some variation between male/female and also species) then it can arm the laser?
If such an array doesn't give you enough accuracy perhaps replacing it with a LIDAR module? That would necessitate the running of further processing (essentially using the point map to discriminate objects of human size and generating an angular "no-fire" zone in order to always keep such objects inside those angles?
I've been following this with great interest, since this device if it's made sufficiently accurate, safe and effective could potentially see use in many different applications.