DRVR v2 is now making it around the park on its own (with just a few nudges from me). The loop is taking about an hour and luckily that's just about the limit of the power of the 3500mAh 18650 battery I'm using.

The video above (about 10x realtime) shows the camera image and a colorized version of the segmented image. The robot is trying to keep the grass at a consistent distance from the left edge about 3 feet in front of it.

• Green is grass
• Red is the path
• Blue is a bench
• The horizontal white line marks a point about 3 feet in front of the robot
• The marks the range of legal values for the grass on the left of the robot
• The L, R, S in the upper right of the image is the predicted action to turn left, turn right, or go straight.

The path planning is fairly simple at the moment. It tries to keep the left edge of the grass between the two vertical lines by turning left or right in small increments. When it is between the lines it goes straight.

There's a few interesting bits in the video:

At 0:39 DRVR approaches the bench. 

At 0:41 The grass is at the right place, but the code notices the bench and turns right until it's in the valid range.

Starting at 1:06 the algorithm falls apart because we're going uphill and the target line is above the horizon. I should angle the camera down to avoid that and perhaps move the target line closer when the robot is on an incline. It got stuck and I had to manually give it a few instructions.

At 1:47 there's a different (similar) path planning problem. The left edge of the grass is no longer visible 3 feet in front of the robot. It's locking on to the grass on the far side of the path and spinning to the right. That is going to require some smarter rules to understand the geometry.

I'm moving the RVR base in small increments (3 degree turns, 1 foot movements). That seems to be difficult for the RVR. The Pi sometimes tells it to turn and nothing happens. It accumulates the desired heading and eventually the RVR makes a larger turn and catches up to the desired position. I believe I'd be better off if I calculated the correct heading from the image and told it to turn to that. That would consolidate several small turns into one larger one and avoid some of the static friction.

Overall I'd call this a successful run. It's way slower than the previous generation robot, mostly because it is trying to steer a tighter path and moves in 1 foot increments instead of 3 foot. The Oak-D should be able to process the data in real-time, but since it is taking pictures each frame (to make the video you saw above) there are delays to avoid camera shake and to write the images to the SD card. I'm sure I can improve the performance.