At this point my research has led me to ROS modules RTAB-Map which will create a three dimensional representation of a robot's environment. It seems very promising... but building such a representation is only half the battle. How would a robot make good use of this data? My research has not yet uncovered applicable solutions.

The easy thing to do is to fall back to two dimensions, which will allow the use of standard ROS navigation stack. The RTAB-Map ROS module appears to make the super easy, with the option to output a two dimension occupancy grid just like what navigation wants. It is a baseline for handling indoor environments, navigating from room to room and such.

But where's the fun in that? I could already do that with a strictly two-dimensional Phoebe. Sawppy is a six wheel rover for handling rougher terrain and it would be far preferable to make Sawppy autonomous with ROS modules that can understand and navigate outdoor environments. But doing so successfully will require solving a lot of related problems that I don't have answers yet.

We can see a few challenges in the picture of Sawppy in a back yard environment:

• Grass is a rough surface that would be very noisy to robot sensors due to individual blades of grass. With its six wheel drivetrain, Sawppy can almost treat grassy terrain as flat ground. But not quite! There are hidden dangers - like sprinkler heads - which could hamper movement and should be considered in path planning.
• In the lower right corner we can see transition from grass to flat red brick. This would show as a transition to robot sensors as well, but deciding whether that transition is important will need to be part of path planning. It even introduces a new consideration in the form of direction: Sawppy has no problem dropping from grass to brick, but it takes effort to climb from brick back on to grass. This asymmetry in cost would need to be accounted for.
• In the upper left corner we see a row of short bricks. An autonomous Sawppy would need to evaluate those short bricks and decide if they could be climbed, or if they are obstacles to be avoided. Experimentally I have found that they are obstacles, but how would Sawppy know that? Or more interestingly: how would Sawppy perform its own experiment autonomously?

So many interesting problems, so little time!

(Cross-posted to NewScrewdriver.com)