I made it in a hurry and was able to produce it in three hours! I think even an inexperienced person can make it in half a day to a day.

Revamped mobile app for management and integrated with ChatGPT. Multiple mowers can be managed with free speech.

And if you have a simple question other than robot management, they can answer it!

Someone who see Roktrack in action for the first time say, "If there are no pylons, where would it go?".

However, this does not happen. If there is no pylon, it simply turns 360 degrees on the spot and stops because it cannot find the pylon. If you look at mowing through a polygon that is vertexed with pylons, you will see that the pylons are used to define the boundaries. But in fact, Roktrack uses pylons not so much to define boundaries, but rather as guideposts to point the way to where it should go.

Basically, Roktrack heads toward the pylon it see, and when it gets a certain degree closer, it turns around and find the next pylon, and continue in the same manner. By gradually shortening this approach, it moves inward and fills the polygon. Since the principle of operation is simply to approach a visible pylon, it is impossible for Roktrack to go in the wrong direction unless it is misdetected. Personally, I find this more reassuring than GPS guidance because the working area is visually visible.

Drive Route

Flow Chart

*The complete flowchart is uploaded as a pdf file.

Encoderless

Pylons as signposts pointing the way make the drive motors encoderless. As shown in the image below, it is possible to calculate the deviation of the recognized pylon from the center of the image. In this case, there is a 20px gap to the right of where the center of the screen should be (the position of the recognized pylon). In this case, Roktrack will increase the PWM of the right motor and decrease the PWM of the left motor. The same calculation is performed during the next process, and if there is a gap, it is compensated. In this way, straightness is gradually ensured.

It is not where you are, but whether you can see your way.

Because the Roktrack is designed to be used in harsh conditions, such as on rough  terrain or left outdoors, it must solve many challenges. For this reason, a single component is designed to have multiple roles. The following are examples of parts with multiple roles.

Part	Roles
Splash guard	- Protects switches and connectors from rain bounces - Prevents scattering of pebbles by high-speed rotating blades
Crawler cover	- Increase the strength of the track system - Prevent grass from wrapping around rollers
Camera Cover	- Prevent direct sunlight - Prevents rain
String to support the camera	- Reduce camera vibration - Locks the top panel open
Bumper	- Bumpers that sense physical contact - Handle for carrying
Top Panel L-Shaped Angle	- Prevents rain - Fixing Splash Guard
Solar Panel	- Solar Charging - Reduces heat to the interior
Nylon cord attached to the blade	- Prevents grass from wrapping around the motor shaft - When the blade falls off, the color stands out and easy to find from the field

Roktrack makes a turn as it approaches one pylon and searches for the new next pylon. While it should preferably turn continuously and without interruption, in practice it repeats the cycle of "turn for 500 ms, shoot, infer". This is because shooting while turning will result in a blurry image as shown in the image below. 

On the other hand, images taken while moving forward on the target object are not terribly blurry.

If you are building a robot with a cheapo webcam, the turning should be intermittent and awkwardly controlled.

In order to detect uncommon and original objects such as pylon(traffic cone), it is necessary to create custom models that are trained on their own. Roktrack uses yolov8 (nano model) to create custom models. The number of images for each class used for training is as follows

Model for mowing navigation

After training, I am exporting in onnx format with 320*320 and 640*640 image_size. The reason why I am exporting with two image_sizes is because of the difference in the time required for each inference. The former takes about 1 second on the Rasberry Pi 3A+, while the latter takes a little over 3 seconds. During actual mowing, I use a light model while the pylon is recognizable, and if it is lost, I use a heavy model to be able to detect objects in the distance. In my experiments, we were able to detect a pylon 50m away when using the 1280*1280 model; it takes about 10 seconds to infer one image, so it is impractical to use it for navigation while moving.

Model for number recognition

The number recognition model is exported with an image size of 96*96 to speed up processing. As explained in previous log, this model infers on cropped images, so low resolution is not a problem.

Models for Animal Detection

This is also exported in 320*320 and 640*640 size, but it leaves some false positives. The nano model can not capture the characteristics of each class; a larger model, such as small, might be better.

I feel that at least 1,500 images per class, preferably 3,000, are needed to achieve satisfactory accuracy.

I finally got my Raspberry Pi Zero 2 W. 

It has the same Conrtex A53 as the 3A+, so I knew it would work, but I just replaced it and it worked as is. I was worried that the clock frequency would drop from 1.4GHz to 1GHz, which would slow down the processing speed, but it is within 1.2 times that of the 3A+, so there is no problem with the mowing operation. Also, the heat generation seems to have decreased. The only thing that bothers me is that the USB input is microUSB, so I have to connect my webcam via an adapter. There is no audio jack, so to make the robot talk, I have to think of a way to deal with it.

It is now possible to monitor people and animals while charging.
When a person or animal is detected, a notification with an image is sent to a messaging application. (Currently, the system is compatible with LINE, a popular messaging application in Japan.)

Detectable objects are as follows.

Person Monitoring Mode

• Person

Animal Monitoring Modes

• bear
• deer
• monkey
• boar
• badger
• cat
• civet
• dog
• fox
• hare
• mice
• racoon
• squirrel

I am excited about the many things we can do by preparing learned models as needed. I would like to add functions such as a simple patrol robot.

Roktrack can basically mow the weed with just a pylon(traffic cone). However, if the pylon happens to be on the neighbor's house, or if a truck happens to be passing by with a pylon on it, the mower will head in that direction. Digit OCR was designed to address these issues. To achieve this additional functionality, all that is needed is to attach a piece of paper with a single digit number printed on it to the pylon. The robot identifies individual pylons by recognizing this number. This means that it can select the pylon it should aim for among the pylons in its field of vision. This inhibits unexpected movements.

Why OCR? Generally, AR markers and QR codes are used for robotic applications. To use these, it is important to be able to detect edges. However, in reality, the edges of markers are blurred when trying to detect them from a distance of 10m or more. I have come to the conclusion that it is difficult to identify a marker from a distance of more than 10 meters with a wide-angle camera. In order to detect objects with blurred details, object detection, which can use more ambiguous features, was suitable. In order to increase the detection distance and speed up the processing, the following is processed.

The image is captured by the camera at the highest resolution. The resolution of that image is then reduced to perform inference in the pylon model. The detection content is normalized and the corresponding portion of the original highest resolution is extracted. The extracted part is then inferred by the Digit OCR model with a resolution of 96*96 to identify what numbers are pasted on the pylon. And since the existing OCR model for transcribing text takes several seconds to infer, I created a very simple custom model that can collect images of numbers and detect only the digits 0-9.

Reference

AR Marker shot from 10m away

AR Marker shot from 20m away

AR Marker shot from 30m away

This would make edge detection impossible, so AR markers would be limited to detection at a maximum of 10m

"8" taken from 30 meters away

For simple numbers, object detection can detect objects as far as 30 meters away. If the resolution can be increased to FullHD, 2K, 4K, and 8K, it will be possible to identify individuals at even greater distances. (This is if the data can be transmitted from the camera to the computer without delay.)

Rainproof

L-shaped angles are affixed downward to the top panel to prevent rain from the upper direction. At the rear of the body are components that should not be wetted by water, such as air intakes, power switches, and charging connectors. On a strong rainy day, the splash guard protects them from being hit on the ground and bouncing rain on these. To prevent insect infestation, sponge tape is applied to the tops of the front, rear, and side panels to make them airtight.

Against condensation

If the robot is left outdoors, condensation will form inside the robot in the morning on days with large temperature differences. When charcoal is placed inside, the charcoal absorbs excess moisture and condensation no longer occurs.