The earlier method of glare removal was too complicated and the processing was taking too much time. In addition, the use of SURF algorithms had some disadvantages:

• They are not free/open algorithms and require extra licenses
• They distort the images through an affine transform
• R/G/B channels are distorted differently and hence the corners of the images (where lens distortion is higher) showed channel mismatches, like the image we shared earlier (shown below).

A little "out of the box" thinking made us realize the following about the spots:

• They're always in the same location on the image
• They would usually cover the macula or the central retina (a small portion of an otherwise much larger image of the retina)
• We can and will always take multiple images from the same patient

We also wanted the processing algorithm to be extremely fast. Hence we decided to use opencv inpainting. This method allows us to segment parts of an image out and interpolate the "hole" from the surrounding parts of the image to get a clear image. Also, because we know that the bright spots only lie in the center of the image, we could only perform these image processing operations on a small central "region of interest" of the image, drastically reducing the number of pixels that need to be processed. The result is shown below:

Here's our input image, after processing to extract only the "fundus" section:

And here's the same image after inpainting the glare spots (and playing around with an optimum dilation kernel, which was found to be 25x25px):

It does a great job! The code (all in python) is available here on our github repo. As can be observed, for the purposes of an examination by a clinician, it's perfect. Also, it's all in python and can be easily integrated with the rest of our code. It also uses all opencv core functions, which are covered by the BSD license.

The code took 8ms to process this image on a macbook pro (13" 2015 model) and 71ms on the raspberry PI model 3B... not fast enough for live processing, but definitely fast enough for processing post-capture!