So after some testing and Reading other methods of driving LEDs. I decided on Converting the output voltage to Current Source. That it would be the best way to Linearly display the output voltage. While my original idea did work to drive the LEDs they would Cut to zero Brightness(Lumis) still at 500mv output. That is because controlling it by voltage cannot be completely linear Due to the LED forward voltage. Now that I am controlling them by current we are actually able to accurately Display the voltage on a Blue and Red LED. Now that I solved that problem. I'll Re Design the PCB too incorporate these changes. It was pretty fun finding a fix for this.