Eye Controlled Wheelchair!

Steve tested my new version of the pc software, 4d, and to my astonishment he actually liked it! No major complaints. Amazing. This is about the fourteenth iteration of the software, and finally we have one that is easy to use, accurate, intuitive and safe. With one caveat however - until we sort the centring issues related to the flawed hand design, we can't fully evaluate 4d. To show 4d in action, here's an update video.

I was excited when I saw that footage. It is now getting pretty close to my original vision of how it should be. I wanted to make a system that would perform as well as a unmodified chair, controlled by a non disabled hand. And this is pretty nearly there. Steve's movement in this footage is smooth and controlled - I can't wait to have a go myself...

Mk5 electronic hand

This is the 3d design of the new hand, intended to overcome the centring problems we've had with all the previous models. Rather than two sets of jaws, each pushing the joystick shaft indepedently, this new design has only one arm actually touching the joystick. That arm, and it's accompanying servo, are then mounted on the second arm. I'm hoping that this configuration will eliminate the problems inherent in the current system. I'm printing the first Mk5 at this very moment, and hope to get the resulting prototype down to Steve for testing by early next week.

We've got our test pilot back! Steve has finally got over the pneumonia sufficiently to be able to resume his role as test pilot. While he had been dodging his eyedrivomatic duties, i had done a lot of software development, and was supremely confident that software version 4c was the final version. Steve, over the last five days has proved me entirely wrong. So I have been feverishly writing code, which has resulted in version 4d. Unfortunately, the testing has highlighted a more serious problem with the hardware - or more specifically, the electronic hand. The basic design is flawed. At the moment there are two double sided arms at right angles to each other. They move independently, each being controlled by a servo. The joystick then slides between the arms. In order to slide freely, there is a certain amount of slop in the system. This slop, it turns out, introduces an unacceptable degree of inconsistency into the self centring mechanism.
So, next week's task will be a major redesign of the electronic hand.

Steve hasn't recovered enough to help with the testing yet. I've been doing my best, but I have few opportunities. Software development is going well however. I am cautiously optimistic that the software is now feature complete. I have successfully (I hope) addressed all the problems we'd had with previous versions of the software. It needs thoroughly testing of course. Here's a screenshot of the main driving grid.

The guiding principle of eyedrivomatic is safety. Controlling a wheelchair with eyegaze is potentially dangerous because of the inherently temperamental nature of eyegaze. The equipment can lose track of your eyes easily, particularly in fluctuating lighting conditions. This can occasionally cause the mouse to jump about the screen, or stay in one place. Irritating if you're writing a letter, potentially dangerous if you're driving your chair. Overcoming this hazard was my first job as designer.

What I did was the following: To move the chair you click one of the direction buttons. The chair then moves in that direction for a set number of seconds, and then stops. In order to carry on you have to push the continue button or a different direction button. If you push the same button twice in a row, nothing happens. When you've pushed the continue button, the chair will continue in the same direction for another duration. Then you have to push the direction button again. And so it continues – direction, continue, direction, continue, direction etc. This way, the chair cannot get out of control for more than a few seconds, before it stops on it's own. If you push the next button before the previous duration is over, then the chair continues forward (or turns) smoothly, without stopping. Nice and safe. Well, as safe as I can make it anyway. Any electric wheelchair can be dangerous, no matter how it's controlled.

The 'electronic hand' has two servos, one controlling forward and backwards movement, and one for left/right. Movement duration is user selectable for each servo. Overall speed is user controlled as well, and there is a button to toggle between driving and manoeuvring modes. A stop button of course, and on chairs that support it, there's also a panic button that turns the entire wheelchair off.

Currently the software only works in conjunction with the Grid2 AAC package. Once the current software version has been proven in the field (fingers crossed), I will make a stand alone application for pc and one for mac. Meanwhile I've put all the files necessary to build it up on the github repository. No instructable yet, but watch this space!

Steve is out of hospital now, and over the worst of his pneumonia. ALS makes you fragile however, and Steve is still a bit broken. He's not up to doing any testing yet, so, with the deadline looming I've taken over that role. Straight away I noticed a problem... The mk2 electronic hand attached to the joystick unit with velcro. This was inadequate because, as the user turned left or right, the hand would twist a little bit out of alignment. This meant that the chair then wouldn't go in a straight line. Steve came up with the idea of a locator plate, that would fix permanently to the chair, and that the hand would then sit on - locating it perfectly every time. When I came to fit my own 'Evans Locator' I found that it was hard to align the plate perfectly. So I have made the Mk2 evans locator plate. This one is adjustable, even after being fixed in place. I've printed it, and will fit it to my chair this morning. Then we will be back in business.

Being an inventor with ALS can be frustratiing. To perfect Eyedrivomatic I need to make small changes to the software, then test it - then make more changes and test again, and so on. The problem is that my hands don't work. I can't put the electronic hand on myself, and often I don't have a carer available to help me. I can't leave the electronic hand in place because I use the wheelchair joystick as a mouse to control the computer. This is why the partnership with Steve worked so well - because he does have a full time carer, and can test whenever he likes. Frustrating.

Mk2 Evans Locator Plate. (prior to assembly)

Mk3 Electronic Hand (note the four holes in the base plate for the prongs of the Evans Locator Plate)

Things were going very well until last week.... We are now on the fourth and final version of the software, which is rapidly approaching beta. The hardware is ready for beta testing now, and we had hoped to be recruiting testers from the MND/ALS community this week. But last week Steve got ill.

ALS kills by weakening the sufferer's breathing muscles to the point where you are unable to breathe well enough to clear the carbon dioxide from your lungs. You essentially suffocate, with your family around you, powerless to help. Steve and I are both approaching that stage. Our ability to breathe is adequate, but only just. When you are balanced on that knife edge, your health becomes a fragile thing, and any illness can quickly become serious.

Last week, Steve got a chest infection. A minor thing that a healthy person wouldn't even miss work for. After two days Steve was rushed to hospital unable to breathe. Since then, for the last week, Steve has been in and out of intensive care. This morning he told me that he has been moved on to a general ward, and the doctors say he will be leaving the hospital in his wheelchair - not a box. The relief was incredible. I cried.

Steve has become a really close friend, and an essential part of the Eyedrivomatic team. Without him I would be broken. Eyedrivomatic would have continued, but without Steve, only at a fraction of the speed. It got me thinking too. My health is as fragile as Steve's, but if I die Eyedrivomatic would stop. All the files are on my pc and nowhere else. Anyone who wanted to carry on the project would struggle without the files, and good documentation. So, in the short term, software development has been put on hold, while I get everything properly documented and up on our github repository. More news next week

Wow! Things are moving fast here. After my successful test of the eyegaze only version - eyedrivomatic pure, we decided it was time to test it in the real world, on someone who genuinely has only eye movement left. Luckily we have just such a person on our team - Steve. With the help of my carers, I built Steve an eyedrivomatic system, and posted it to him. This is the video of his first go.

The test highlighted a lot of shortcomings with the software, so thats the next job for team eyedrivomatic - a big software rewrite. When we've nailed that down, we'll put all the files up and do an instructable. There are thousands of people who would benefit from eyedrivomatic now straight away, and I want to help them all as soon as possible.

Following last week's successful test of the switch version of eyedrivomatic, we are moving over to concentrate on the eyegaze only version. We are calling this 'eyedrivomatic pure'. This is what most people will think of when they hear 'eye controlled wheelchair', so I am busy rewriting the software. Also we have some hardware developments to report, the Mk3 brain box and the Mk2 electronic hand. The improvements to the brain Box are all aimed at making it easier to build. The electronic hand however, has had a complete redesign. It is much smaller (micro servos rather than standard), is easier to build, and uses less plastic. Photos to follow.

eyedrivomatic test number one