The vision system now generates a disparity map of the two webcam pictures (3D picture).

The robot adds near parts of this 3D picture to my 3D points and can see "more" objects.

If a threshold of objects in a set distance is reached the robot detects it as a obstacle. But how high should this threshold be set? I wrote some functions which do learn that. I did set the start thresholds very low, so the robot drives backwards to avoid objects. There is a "did drive backwards counter" to note this. If this counter reaches a set value the robot will increase the objects threshold. And the counter is set back to zero.

After some time "learning" the robot should drive forwards and avoid driving into objects. If the robot is  stuck somewhere the objects threshold is decreased. The accelerometer is used to measure if an object was touched.