This program, written in C++, uses the open-source libraries OpenCV and OpenAL to get the depth data from the simulator (or hardware depth camera) and convert it to audio. See back end log for details.
This shader is a script that runs on the graphics card, and reads the depth buffer from the graphics card and turns it into color pixel output. See simulation log for details.
Our C# script that runs in Unity, calls our shader to turn depth buffer to color, then reads the color pixels into RAM, and calls the DLL plugin to get that data to the audio generation software. See simulation log for details.
This is the main source code from our Unity DLL plugin that copies the depth data from our Unity C# script and provides it via a shared memory interface to our audio generation software. Copied and modified from the Unity graphics plugin example. See simulation log for details.