Raspberry pi tennis ball tracking robot
Already have an account? Log in.
To make the experience fit your profile, pick a username and tell us what interests you.
raspberryPetDog can operate in both manual automatic mode.
The robot uses image processing algorithms in automatic mode to track and follow a tennis ball.
You can also use the manual mode to control the robot using a wireless keyboard.The instructions to move the robot manually are available in the github page.
In this post, I try to explain how the robot tracks the ball using image processing algorithms.
You first need to install OpenCV in your Raspberry Pi which you can find the tutorials in following websites:
Install OpenCV and Python on your Raspberry Pi 2 and B+
Install guide: Raspberry Pi 3 + Raspbian Jessie + OpenCV 3
sudo apt-get install python-wxgtk2.8 python-matplotlib python-opencv python-pip python-numpy sudo apt-get guvcview
You can use both cv and cv2 libraries but I used cv2 for this project.The petdog.py code tracks a tennis ball using color detection methods and calculates center of mass of the tennis.
To capture a video, you need to create a VideoCapture object. Its argument can be either the device index or the name of a video file. Device index is just the number to specify which camera is used and in this case it's zero because there is only one camera module.
Milad
cap = cv2.VideoCapture(0)
Next you can use Property identifier to modify the height and width of VideoCapture object.
cap.set(3,640)
cap.set(4,480)
3- Width of the frames in the video stream.
4- Height of the frames in the video stream.
The VideoCapture object needs to be read frame by frame to perform object detection which is done using .read method:
_, frame = cap.read()
Now you need to convert image frames from one BGR color-space to HSV color-space.
hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
Next you need to use upper and lower color thresholds to define range of yellow color in HSV which is used to find tennis ball. The values needs to be changed based on light settings.
lower_yellow = np.array([30, 255, 135], dtype=np.uint8)
upper_yellow = np.array([40, 255, 185], dtype=np.uint8)
An image mask can now be created to get only yellow colors. The image mask is used to calculate image moment of tennis ball which is a certain particular weighted average.
mask = cv2.inRange(hsv, lower_yellow, upper_yellow)
moments = cv2.moments(mask)
The image moment can also be used to calculate area and center of mass of the tennis ball.
area = moments['m00']
Now we need to check the area of the object before calculating the x and y centroids.If the object is too small then there is a high chance that it's not a tennis ball.
if(area > 100000)
Calculate x and y centroids:
x = moments['m10'] / area
y = moments['m01'] / area
And finally we can implement the function that calculates robot movement based on changes in x and y centroids values.
def move_pet(x,y,area):
if x > 330:
pivot_right((x-320)/(320*12))
elif x < 310:
pivot_left((320-x)/(320*12))
elif ((310 <= x <= 330) and area<1500000):
forward(0.05)
elif ((310 <= x <= 330) and area>4000000):
reverse(0.05)
We know the center of each image frame is 320 because the total width of each image frame is 640. The robot moves after each movement so the tennis ball stays at the center of image frame.
The total time for pivoting or turning the robot in each direction can simply be calculated using:
(x-320)/(320*12)
You can also use the calculated area to move the robot back and forward as well as turning which you can see in the video.
For example you know the tennis ball is too close to the robot and the robot needs to move back in the following case:
elif ((310 <= x <= 330) and area>4000000):
reverse(0.05)
Or move forward the robot petdog if tennis ball is too far:
elif ((310 <= x <= 330) and area<1500000):
forward(0.05)
Now you can use the following commands to start the robot:
sudo modprobe bcm2835-v4l2
sudo python petdog.pyThis includes the overview of object tracking code and you can go ahead and create your own raspberryPetDog robot. I hope you create your own raspberryPetDog robots with even fancier features.
In this post, I will go over details to drive the robot manually using keyboard and command line.
I won't go over the details of setting up raspberry pi Raspbian OS as there are many good tutorials and it's not the main point of this project.
Once you download Raspbian OS and setup the OS then you can go ahead and power your raspberry pi.
Once you login automatically then you can go ahead and connect to your wi-fi network.Once you are connected to internet then you can enable SSH and camera module.
You can enable SSH and camera module using Raspberry Pi Configuration if you use GUI and following command if you use command line:
sudo raspi-config
Next update your system's package list by entering the following command:
sudo apt-get update
And upgrade all your installed packages to their latest versions with the command:
sudo apt-get dist-upgrade
Next find the IP-address of your pi using following command:
sudo ifconfig
As you can see my IP-address of raspberry pi is 192.168.1.85.
Now go ahead and download robot_cmd.py from github repository of this project in your computer.
This code lets you drive the robot using keyboard keys through SSH.
I use PuTTY as a SSH client because I use a windows machine which you can find a complete PuTTY tutorial on it on raspberry pi website. You can use fileZilla to transfer downloaded code from github repository in your computer to your raspberry pi as it's shown below.
You can now run robot_cmd.py using following commands:
sudo python robot_cmd.py
Now using following table as controller guide to drive the robot:
| Character | Action |
| w | Move forward |
| s | Move reverse |
| a | Turn Left |
| d | Turn Right |
| q | Pivot Left |
| e | Pivot Right |
| x | Exit |
Double check that your robot behave according to the guide otherwise you may need to switch wires of your DC motors connected to L298N dual H-bridge module.
Finally, I go over the code and explain how it works:
def init():
gpio.setmode(gpio.BOARD )
gpio.setup(7, gpio.OUT)
gpio.setup(11, gpio.OUT)
gpio.setup(13, gpio.OUT)
gpio.setup(15, gpio.OUT)
gpio.setup(16, gpio.OUT)
gpio.setup(18, gpio.OUT)
gpio.output(16, True)
You first need to set gpio mode to the Board to use GPIO pin numbers using following:
gpio.setmode(gpio.BOARD )
Now you setup GPIO pins as output and enable pin #16 to enable DC motors.
gpio.setup(7, gpio.OUT)
gpio.setup(11, gpio.OUT)
gpio.setup(13, gpio.OUT)
gpio.setup(15, gpio.OUT)
gpio.setup(16, gpio.OUT)
gpio.setup(18, gpio.OUT)
gpio.output(16, True)
The GPIO pins needs to cleared and reset at the end of each action by setting up status of GPIO pins to false:
def clean_up():
gpio.output(7, False)
gpio.output(11, False)
gpio.output(13, False)
gpio.output(15, False)
gpio.output(16, False)
gpio.cleanup()
Now I explain the functions to move forward and reverse:
def forward(tf):
gpio.output(7, False)
gpio.output(11, True)
gpio.output(13, True)
gpio.output(15, False)
time.sleep(tf)
clean_up()
GPIO pin numbers 11 and 13 are set to True as they are responsible for moving DC motors forward.The pin numbers 7 and 15 are set to False as they are responsible for moving DC motors backward or reverse.
The input parameter tf defines how long the motors will turn.
time.sleep(tf)
time.sleep(time) function defines how long the motors have to turn before stopping using following clean_up function.
The function to move the robot backward works the same way except you set GPIO pins 7 and 15 to True and GPIO pins 11 and 13 to False.
def reverse(tf):
gpio.output(7, True)
gpio.output(11, False)
gpio.output(13, False)
gpio.output(15, True)
time.sleep(tf)
clean_up()
Now how does the turn_left function work? Well,That's easy! You just enable GPIO pin #13 as it turns forward both DC motors on the left side of robot.
def turn_left(tf):
gpio.output(7, False)
gpio.output(11, False)
gpio.output(13, True)
gpio.output(15, False)
time.sleep(tf)
clean_up()
The pivot left function pivots the robot or turn the robot around z-axis counter clockwise.To achieve this,the...
Read more »In this post, I will go over instructions to setup and put together the Robot Platform.
First let’s start with components list:
The first step would be mounting 4 DC Motors to 2 side plates and then connecting the lower plate to side plates. Once that is done then you can go ahead and mount the L298N dual H-bridge module to the middle of lower plate.
The next step would be connecting the DC Motors to the L298N dual H-bridge module as it’s showed in following picture:
Each DC motor has two wires for rotating in each direction (One wire for turning the motor clockwise and one wire for rotation in counter clockwise direction). The forward wires of motors on the left side needs to be connected to out1 port of L298N dual H-bridge and the reverse wires are connected to out2 pin. The same goes for the right side of robot and forward wires are connected to out3 port of L298N dual H-bridge and the reverse wires are connected to out4 pin.
Next the rechargeable battery should be mounted at the back of lower plate as it’s showed above. Once the battery secured then you can go ahead and connect the +12v and GND wires of the battery to the VCC and GND inputs of L298N dual H-bridge module. A power switch and a charging connection needs to be used as it’s shown below in case you would like to use a reachable battery.
The GND and +5V of L298N dual H-bridge module need to be connected to pin #6 (GND) and pin #2 (+5V) of raspberry pi. I decided to use a TriBorg GPIO replicator board as it gives me more power pins and it replicates GPIOs.
The next step would be connecting the input pins of L298N dual H-bridge module to the GPIO’s of raspberry pi in following order:
You will need to look at raspberry pi pinout map to make sure pins are connected correctly.
ENA and ENB pins are used to enable and disable Outputs of L298N dual H-bridge.
Now we can go ahead and mount the top plate, front plate and back plate of Robot Platform.
This covers the setup of Robot Platform and you can visit Sainsmart website for more details on setting up the Robot Platform.
In next post, I will go over the driver code and explain how to move the robot using a wireless keyboard.
The robot can track tennis ball at the moment.
Jerry Cheng
Max.K
BTom
Arielle