Eye-focus

Description

The aim is to make a sensor that will respond to the human eye doing accommodation (i.e., focusing) and driving some external optics accordingly. I plan to build one into my microscope, but it can potentially be built for other things that need focusing, e.g. binoculars, cameras, maybe even adjustable eyeglasses.

Details

The aim of the project is to make the microscope smart. That is, it will refocus automatically depending on what you want to look at. It will do this by detecting your eyes trying to refocus, which occurs subconsciously. So, it's almost like a connection to one's mind.

The neat thing about this project is that the technique can be applied to a variety of devices, not just microscopes. Also, there is another closely related idea, which I don't plan to do now: by sensing the direction the eye is looking to, it is possible to shift the microscope laterally, kinda autoscrolling.

I plan to put a sensor into an eyepiece of my microscope. The sensor is to detect the focusing state of the eye, and based on that an Arduino will drive a stepper motor attached to the focusing wheel of the microscope.

The sensor is to be based on red-eye effect. Consider an LED shining into an eye. If the eye is focused, a small area of retina corresponding to the optical image of the LED will be illuminated. That area acts as a secondary light source, bouncing some light back out of the eye and focusing it almost exactly back into the LED. This return light, the red-eye light, can be intercepted by placing a semitransparent mirror in the path.

If the eye is not focused on the LED, a) the image will blur, reducing its intensity, b) the image won't be refocused on the detector, reducing the signal even more.

Next step is to quickly sweep the focusing back and forth, in a search for a peak of the red-eye effect, indicating the focused position. The easiest way to do it is to move the LED-photodiode structure back and forth, since the eye focusing range collapses into mere 5 mm when it is behind a strong lens of the eyepiece.

Project Logs

Collapse

infrared
DeepSOIC • 03/07/2015 at 14:01 • 0 comments

Well, while I abandoned the project, I still have something to share.
First of all, I was busy studying my 3d printer. After realizing that the familiar SketchUp doesn't suit my needs, I switched to other CAD, and it happened to be FreeCAD.
FreeCAD is an open-source (LGPL) parametric geometry modeler, and it's super awesome because it's open-source. I've jumped in, and programmed support for Snell's law into its main 2D profile editor named Sketcher. That can seriously help with optics design. I'm very proud for that my Snell's law code was merged into main development branch of FreeCAD!
Also, I have set up some gear to aid me with infrared. The gear is just a power IR LED to light my bench, and a smartphone with a hacked camera that is able to see infrared (I just ripped IR cutoff filter off the camera).
With that setup, I made an important conclusion. All polarizers that I have (most are from LCDs, but also polarizing sunglasses and a photographic filter) are completely ineffective at near infrared (850 nm)! They just don't do anything, they are simply transparent. Surprizing, isn't it?
This means, I have to find some obscure polarizers that are specifically made for NIR, probably very expensive stuff. =(
Here are some pics made in infrared light. Some are out of focus, because I brke the focusing of smartphone's camera during surgery, and it is now fixed at focus distance of about 150 mm.
project abandoned
DeepSOIC • 08/18/2014 at 13:54 • 0 comments

I have to admit I have abandoned the project. After the experiments I've done, I feel it's impossible to put the eye-accommodation sensor into the eyepiece. The problem of lens flare (reflections), coupled with lowness of red-eye backscatter and ridiculous size constraints imposed by space available in the eyepiece, all that makes me feel the project is hopeless. I may make another go at it in the autumn, but I'm not sure.
a setup with webcam
DeepSOIC • 07/12/2014 at 17:27 • 0 comments

So, this is the first time I was able to achieve a red-eye through an eyepiece! TADA!
That bright blob is strong as hell - it's the reflection off eyepiece's lens' surfaces.
But at the very least, the effect is there - it's in infrared, behind the eyepiece, and dependent on focusing (although a bit more playing is needed here). Good news!
culprit is found? -no =(
DeepSOIC • 07/06/2014 at 19:06 • 0 comments

I was using pieces from CDROM heads for beam splitters. After a bit of googling, I've found some of them are actually polarizing beam splitters, which I was not expecting. While they are potentially very useful stuff, in my case their polarizing properties can be the reason I can't get any positive results.
UPDATE. After a bit of mucking around with polarizers, I figured out that the mirror is not at fault. While the transmitted light is indeed very polarized, the reflected one is not, I didn't notice even a tiny bit of polarization. So, while the mirror is not optimal, it is far from being the culprit.
having fun with my new 3d printer
DeepSOIC • 06/25/2014 at 21:07 • 0 comments

it's just awesome!!! It is PrintBox3D (aka 3dadget), Russian!
3D-printing small things is a bit of an art. On the picture (left to right)
SOIC package (for scale) ; a good printout ; a not-so-good printout
sorry for the lack of progress,
DeepSOIC • 06/24/2014 at 22:46 • 0 comments

But there is really almost nothing to report of. Except that I have modified a couple of cameras so they are sensitive to infrared and verified that the red-eye effect is observable in infrared. It's still unclear, what wavelength I'm using, but I've ordered a bunch of IR leds for which the wavelength is specified. And a bunch of photodiodes and an amplifier.
I also plan to do some experiments with an image sensor instead of two photodiodes, to get some extra info.
tired of gluing random stuff together...
DeepSOIC • 06/17/2014 at 22:17 • 0 comments

... and waiting for a new 3d-printer to arrive. I can't live without one anymore, when every second hacker here seems to have one =)
idea on sensor
DeepSOIC • 06/15/2014 at 17:44 • 0 comments

Up until now, most photocurrent I was getting from the phototransistor was because of the light bouncing off the lenses of the eyepiece, and probably a fair amount from reflection of the eye's sclera (the white stuff) and skin. I think these "parasitic" reflections completely obscured the signal from the red eye (obviously they are dependent on the position of the sensor). And I have a few thoughts on how to overcome this.
First. The red-eye return light is focused, while the parasitic light is not. So, if I place another photodiode close to the main one, it will receive a nearly identical parasitic light, but not the red-eye return signal. So, subtract the signals from those two photodiodes - and the signal from parasitic light is gone. That's the plan. But I have to switch to photodiodes (not phototransistors) at this point, because sensitivities must match.
Second, I should limit the cone from where the red-eye light is collected to as much close to only the eye's pupil as possible. That's not easy, but since the eye's pupil must be in quite specific position in order to see through the microscope, that is doable.
on IR red-eye
DeepSOIC • 06/07/2014 at 22:33 • 0 comments

After a really lot of googling, I have found some info. I don't know the part number of the optointerrupter I'm using, but 3 randomly picked datasheets were for ones working at 940 nm. Thanks to the info on this page, I know that this wavelength falls onto the onset of eye's opacity. That shouldn't be a problem, but shifting to 850 nm is desirable. It seems that most eye-exploring istruments work at 850 nm.
Second is that the red-eye peak seems to be small enough to be completely obscured by the changes in reflections off lenses and white portions of the eye caused by movement of the detector. So, I probably have to get rid of them somehow... Maybe, doing some calculations will help to figure out by how much I have to suppress unwanted reflections.
close to losing hope
DeepSOIC • 06/06/2014 at 21:59 • 0 comments

The new sensor just does not seem to get anything. As before, everything is brilliant on lens+palm system, but des not work completely with the real eye. Something must be completely wrong, I have no idea what exactly =(
Update: there might be a small chance that some of the structures of the eye that are transparent to visible light are not transparent in the near infrared that I'm using. Or the infrared light is not backscattered by retina. So, I'll probably try making a sensor based on visible light.
Anyway, I wont be able to work on the project during next week, but I'll continue thinking.

View all 23 project logs

Discussions

naroom wrote 08/24/2014 at 01:20

Aww, sorry to hear this one didn't work out. My wife and I have been following it (we're both in microscopy). Thanks for sharing it though! It's a very cool concept, and I hope you get back around to it when you find the right approach.

Are you sure? yes | no

DeepSOIC wrote 08/24/2014 at 12:17

Thanks for support!

Are you sure? yes | no

Eric Tsai wrote 07/24/2014 at 07:51

Very neat project. Best of luck.

Are you sure? yes | no

razorfish_sl wrote 07/13/2014 at 07:39

HI,
Perhaps eye focus is not the way to go, the eye lens will be continually trying to fight the system.( as your system physically adjusts the microscope focus, the eye lens is attempting to focus outside of the control structure.)
It might be better to measure and compensate the fixed system focus, I.E microscope lens path to sample.

If you get the image focused on eyepiece lens , then the eye will compensate for the eyepiece to retana. ( maybe project a cross/dot down to the surface of the sample, then use the software to 'focus' the returned cross/dot)

Are you sure? yes | no

DeepSOIC wrote 07/13/2014 at 08:34

Hi! That fight will likely happen. But I believe it can be tamed just like opamp loop instabilities can be tamed. The problem now is just in sensing the eye's focus - I just cant get it to work =(
What you are talking about is just like an autofocus of a camera, which is certainly not bad, but it will only focus the center of image. That's far from being comprehensive, and it might behave wildly in the absence of an object. But it should be relatively easy to implement - I can just buy a cheap DSLR, take it apart and hack it into the microscope (it might end up being able to take pictures through the microscope as well). Let's call it "plan-C" (I already have a plan-B).

Are you sure? yes | no

Michael R Colton wrote 06/20/2014 at 15:03

Interesting project! Here is are some thoughts (I haven't read all your updates, so if these are covered, I apologize). First, one way you could discriminate between noise and signal is by blinking or otherwise modulating the source. That way you don't have to build another detector, and there are no alignment issues. As for hunting back and forth to find the right focus, you could have multiple detectors, some with optics to detune the focus (say three detectors with -1, 0, and +1 "focus units") so you know that, for example, if the +1 and the 0 detector are reading the same, and the -1 detector is lower, then you are currently at +0.5. That would be hard to align and work to build, but wouldn't have any moving parts.

Have fun! Good luck!

Are you sure? yes | no

DeepSOIC wrote 06/20/2014 at 21:35

Well, the "noise light" I was getting was actually coming from the led (I worked in darkness), so switching it on and off won't help at all. It will help to eliminate the "noise" from the image I see trough the microscope however.
As for the moving parts, yeah... After the mentioned problems with breaking wires, I feel like that is a big problem actually, because it must cover at least 5 mm peak-to-peak, which is kinda lot in the confined space inside the eyepiece.

But at least making it work with vibration and outside of the microscope is essential, I think.

Are you sure? yes | no

Michael R Colton wrote 06/20/2014 at 22:00

Oh, I understand, I read too quickly and thought you meant ambient light leakage.

Are you sure? yes | no

DeepSOIC wrote 06/20/2014 at 22:03

anyway, thanks for the interest. Ideas are welcome.

Are you sure? yes | no

Adam Fabio wrote 06/09/2014 at 02:11

Do a few google /USPTO patent searches - there is some info there. Keep on this project, I'm excited to see what you come up with - Thank you for entering it in the Hackaday Prize!

Are you sure? yes | no

DeepSOIC wrote 06/09/2014 at 23:57

Thanks for support. However, I fell like my project is a little bit in trouble now, since I have built the most-likely-to-work sensor configuration and it doesn't work.

As for patents, I have found a patent on this. The patent is US 7298414 B2. One of the configurations described there is almost exactly what I am trying to do. Unfortunately, not much technical details there, so it does not help me to make it.

Are you sure? yes | no

Adam Fabio wrote 06/12/2014 at 01:36

Don't give up - you may be on to something with the IR transmission / blocking properties of the eye. glass for instance is opaque on FLIR style systems. You still have time!

Are you sure? yes | no

Adam Fabio wrote 05/29/2014 at 04:19

This sounds really cool! Have you looked into Autorefractors? They essentially do what you're talking about focus wise.

Are you sure? yes | no

DeepSOIC wrote 05/29/2014 at 11:00

Thank you, I haven't. It's definitely worth investigation. I'm fascinated by how the medical stuff is so ahead of everything.

Are you sure? yes | no

DeepSOIC wrote 05/31/2014 at 20:46

Well, at least I have found that, yes, autorefractors work the similar way. That is good news, since autorefractors are used to prescribe eyeglasses, so they should be able to measure the focusing quite precisely. The other good thing is that some are working in infrared, exactly what I'm gonna do. Unfortunately, I haven't found (yet) any info on how exactly are they made internally.

Are you sure? yes | no

Eye-focus

Description

Details

Project Logs

Collapse

infrared

project abandoned

a setup with webcam

culprit is found? -no =(

having fun with my new 3d printer

sorry for the lack of progress,

tired of gluing random stuff together...

idea on sensor

on IR red-eye

close to losing hope

Discussions

Similar Projects

Non-contact touch panel

Retro-reflective Bounceless Optical Switch

Simple sun position sensor

Peril-Sensitive Sunglasses

Eye-focus

Become a Hackaday.io member

Just one more thing

Description

Details

Project Logs Collapse

Enjoy this project?

Discussions

Become a Hackaday.io Member

Similar Projects

Does this project spark your interest?

Report project as inappropriate

Send message

Remove Member

Project Logs

Collapse