Getting good low latency video from the robot to the control centre is alway a bit of a condrum. using a webcam connect to a single board computer is usually the way forward. Picture Latency can cause problems when remote piloting leading to collision and crashes (real crashes not software ones!).

We needed good high quality video so we could remotely pilot the robot in various environmental and lighting conditions while being High Definition enough to spot obstacles and hazards in our path. 

https://www.logitech.com/en-gb/products/webcams/brio-4k-hdr-webcam.960-001106.html

After playing we a few streaming solutions, from ffmpeg to RTMP servers, we were seeing latency of 5-10 seconds, after tuning codecs parameters and network/video buffers we could get it lower, but just felt it wasn't real time enough for our robot.  We decided to have a go with NDI for a laugh.

NDI is proprietary codec for HD, low latency streaming over LAN's , however with the right networking equipment it will work over WIFI too ( with a few hacks ) :-).

We built, configured and compiled most of the software from scratch to get a NDI stack running on raspberry pi... it was a long arduous task until we found https://dicaffeine.com/about

This entire distribution turns your pi into dedicated NDI encoder device. You have lots of options for resolution and frame rate.

We terminated the NDI stream to a ubuntu machine running Open Broadcast Studio, which can be configured to receive NDI streams.

After we fixed the network problems with increase data rates of the NDI codec , detailed here https://hackaday.io/project/181449-basic-farm-robot-2021/log/197307-multitech-woes , we got good results with low latency (really insanely quick) HD video from the robot. Error recovery was also swift in case of degraded a wifi connection. Impressive.

You can see all this in action in our youtube video here 

NDI OBS in action

However, as with all things, the real world calls, we found the most of the wifi link bandwidth was saturated with NDI packets, we could of added another 2.4ghz/5ghz router to the robot and another access point on different dedicated wifi channels ,  but this would just of added extra expense and complexity to the robot and the control center. Perfect design is not what you can add to something, but when you cannot take anything more away.

In the end we decided to got for a dedicated 5ghz analogue video connection , they usually have better range and better error recovery, where a 'snowly' picture, is better for the human operator than a complete break in picture feed that many digital encoding systems suffer from.

You can read more about adventures with FPV camera in another log.!

One last thing to mention is the https://dicaffeine.com/about really is great software for your low latency video projects, the free version will run for 30 mins before you need to restart it , it's certainly worth every penny if you plan to used NDI steaming on your network!!!!