SWAN open source AAC

I'd designed the PCB for just the coil part of the electromagnetic interface some time ago (see here). I thought I'd have a go at a 1 inch square test PCB. Keeping the layout simple, a square coil it was, and with the connection pads my trace design came out a little oblong, but the concept would be provable. It's got trace all the way to the centre so that I can experimentally determine the amount of unwanted turns at the axis.

One inch coil

42 turns of 0.01 inch trace

The vertical lines are screen faults on my "weathered" laptop

Unfortunately I've been completely sidetracked by the Arduino, my spare time having been spent using it for martial arts data gathering experimentation (a project which followed the completely useless "electronic candle," see here. Still, you have to make stuff to learn).

The crunch came when I was about to buy some hardware for the new project. Realising SWAN was about to languish for a while (yet again!), I shelved punch monitoring. But I'd gained a modicum of Arduino experience.

Back to SWAN. I'd come across an outfit on ebay (eminence2001) which is set up to produce prototype PCBs using a CNC miller to remove copper, leaving only the traces. I emailed the owner Marek, with the Eagle file attached and a link to this blog. He was kind enough to produce my board very quickly for the princely sum of £5. Here it is, it's a gem -

I've done some quick testing of the PCB. By probing at various points on the coil at 40mA, I determined that the board could actually be single sided, and that the amount of unwanted turns in the middle was rather more than I had expected... In short, it works.

Next I'll hook up the Arduino, and do some proper testing. With the results, the next coil PCB design can be created using Eagle, this time with an "proper" octagonal or spiral coil layout, and additional circuitry. I'll certainly be using Marek's PCB milling service to make the prototype.

He trades as eminence2001 on ebay. His current listing is here

email marek.kuczynski@ntlworld.com

"PCB Printed Circuit Board Prototype Production/Milling/Manufacture/Manufacturing"

Everyone able to construct a word, knows the sequence of the alphabet. During auditory scanning, this is favourable, since the user is aware when the desired letter will be announced. Unfortunately the most commonly used letters are not in "alphabet order". This could be very annoying to a user of SWAN with excellent mental acuity.

letter frequencies, sorted alphabetically

In order to accommodate these folk, it's planned to have an option to scan through the alphabet in statistical order.

letter frequencies, sorted by frequency

Also, I'm experimenting with grouping of letters, to enable even faster selection. Text prediction is also possible, although beyond my software capabilities.

The ability to very simply and cheaply add a switch input to an inexpensive android phone, offers many possibilities for use in areas other than AAC.

A burglar alarm - a passive infra red motion sensor module such as this......

..... costing £0.99 from china, connected with the SWAN electromagnetic interface, would enable a highly sophisticated alarm or room monitoring system to be developed. An app could easily be written in Tasker which could, for instance, send a text message to another mobile phone upon detecting a person in the monitored area. The person receiving the text could then text back with instructions to the android alarm system app -

on - bluetooth an audio file of an alarm sound to a bluetooth speaker.

msg - same with an audio message, e.g. "Leave the premises immediately!"

off - switch audio off

log - send no more text messages, just log times of detection

auto - sound alarm automatically when intruder is detected

Since the PIR module can output pulses >1 sec, the speed issues I'm having with SWAN using MIT App Inventor and Tasker do not arise.

Industrial uses - for instance, a switch at the end of a production line could be triggered as each manufactured item is finished. Connected to an inexpensive mobile phone, data such as line rate, down time, shift total etc. could be sent by text to the production manager at set intervals. The phone could be protected, and made robust enough for industry, by a suitable IP65 enclosure.

I've written the SWAN software using MIT App Inventor. At present, it scans through the alphabet and selection is done by touching the screen, or depressing the smartphone whilst on a foam base (here I'm using the accelerometer to detect rate of change of orientation). Implementing the electromagnetic interface, so that letter selection is done by a commercial AAC switch, has become problematic because of speed issues.

Unfortunately App Inventor just can't do it, and neither can Tasker. For some reason Tasker only registers changes of magnetic field over second in duration, and with all the azimuth calculation App Inventor does, it's not surprising that it's way too slow.

I know that the android OS is fast enough, through Joe Desbonnet's work, and also free sensor apps such as "Magnetic Field Detector" show detection of short pulses well.

Other routes to interface switches were then considered. One solution is to have the phone's charger plugged in during use. The power lead is plugged into an external female micro USB socket. The +5V from the socket is taken to the normally closed switch, then back to a short lead terminated by a male micro USB plug, which is plugged into the phone. In this way, the supply can be interrupted, and this can be detected by the android OS. Unfortunately App Inventor doesn't yet support power monitoring, and although Tasker (the other app development software I'm experimenting with) can do this, the idea is a no-goer because most commercial AAC switches can't handle the possibly high charging current and may not have a normally closed mode. And of course, the flexibility in having SWAN independently powered would be lost.

Another method considered was to use a small piece of aluminium foil taped to the touch screen over a "button". A wire from this to the switch and then to ground could simulate a button press by direct influence on the capacitive screen. After a bit of experimentation, the idea was deemed too dodgy and doomed. Interesting though, and perhaps with the inclusion of a FET it could be made to work reliably. Here's a Hackaday post where the method is demonstrated.

Also considered for use as a switch interface were an ultrasonic transducer next to the phone's microphone, the microphone input jack socket, and the ambient light sensor. All these require more hardware than the simple coil of wire. The electromagnetic interface is the way to go.

So, where to go from here? I'm designing the PCB for the electromagnetic interface. I'm continuing development of the functionality, with both App Inventor & Tasker, but the project really needs a software"plug-in" of some sort, to handle the speed issues. If I can't get any help in this direction, my completed software algorithm could be used by a third party software house as a template to produce some professional code. This would mean I'd have to do a modest "Kickstarter", in order to pay for it. I'll get some quotes, once my algorithm is finalised. 

If I had the time, I'd learn the Android APK development environment, and the whole thing could be done in a professional manner. But it's been way too time consuming just to get to the stage I'm at - I've a modicum of expertise in App Inventor, and just learning Tasker.

The selection of letters from SWAN auditory scanning can be accomplished using the phone's own sensor inputs (i.e. the touch screen, accelerometer or proximity sensor). However, using them may be impossible because of insufficient feedback, or a user's lack of dexterity. It's highly desirable therefore, that an external switch, specifically designed for use by the disabled, can be used with SWAN via a 3.5mm jack socket.

These are called AAC switches, examples can be seen here -

Augmentative and alternative communication switches

more can be found here

To interface these switches to the android smartphone, electromagnetic coupling will be used (details here). A microswitch, for example, can be connected to a coil of wire and a battery via a resistor to limit the current to around 60mA. The coil is placed on the phone, when the switch is closed a magnetic field is generated by the coil, and is detected by the geomagnetic sensor on the phone.
Proof of concept -

9 V battery, microswitch, 150 ohm resistor, coil of 30 turns of 46 SWG wire, cable ties.

Instead of the microswitch, any commercial AAC switch could be incorporated into this interface via a 3.5mm jack socket.
The simplicity of this means that those of a hacking nature should be able to make such an interface with little effort, from materials at hand, or from a scrapyard or dump. I plan to produce a video showing how quick and easy such a build is, with only parts gleaned from a scrap car.

However, a properly designed interface is desirable, for those able to afford it. To this end I've been learning how to use Eagle PCB software. Years ago, I earned a living using acetate sheets, transfers, black tape of various widths and a scalpel. Now, using CAD is wonderful, compared to the "olden days" of tape up. Here's where I'm at with the design of a double sided PCB to (hopefully, if it works - i.e. produces sufficient magnetic field) replace the coil of wire...

One inch coil, 42 turns of 0.01 inch trace. The vertical lines are screen faults on my "weathered" laptop

The PCB will be populated with -

a wound coil (if the copper trace coil doesn't work)
current limiting resistor
>70mAH li-ion battery
MCP73831 or MAX1555 charge management IC (with associated caps)
charge LED
micro USB socket
test push button
3.5mm mono jack socket

In order that the magnetic interface can be used for purposes other than AAC, pads and traces will be on the board for optional components (a transistor & resistors) and connections (to a PIR device for example, see here).

When I first started experimenting (as an android virgin) with a donated HTC Wildfire in February, I constructed a series of apps which graphed the output of each sensor in the phone. Such is the engineer, and these provided valuable steps in learning how to use App Inventor. At this point, using an android phone as the platform for SWAN (a long dormant project, see here) hadn't occurred to me.

The sensor which immediately held my interest was the geomagnetic sensor, found in most smartphones. App Inventor converts the output from the HTC's internal AK8973 3-axis geomagnetism sensor to an azimuth reading (0 - 360 degrees), for utility as a compass. I was disappointed with this, and had hoped that the device's raw data would be available. Anyway, the graph of azimuth vs time proved most interesting.

The trace shows intriguing hunting behaviour when close to active electronics (laptop etc.) and also I was surprised by the sensitivity. A two pence coin (post 1992, bronze to steel) moved near the phone distorts the Earth's magnetic field and the trace moves considerably. At this point I realised I could interface with the phone; I swiftly cut a spare charger lead up, to provide a couple of feet of wire, stripped both ends and, with a 18650 Li-Ion battery ripped from a scrap Vaio power pack, I recreated Hans Christian Ørsted's momentous experiment of 1820. He would have recognised only the wire.

The trace on the screen moved, just as Ørsted's compass needle turned, when current flowed through the wire. So, I reasoned, if you have a small coil of wire fixed onto the phone, over the point where the geomagnetic sensor IC is, then software in the phone can respond whenever current flows though the coil. With only the coil, a resistor to limit current, a battery and a switch, we have a very simple method of wireless on/off communication with the phone.

My first attempt to produce a SWAN device was in 2001, using a microcontroller. My sister bought me PICBASIC software, I made a PIC programmer, a friend got me a handful of PIC 16F84 microcontroller chips, and I began to develop SWAN.

The prototype used a modified computer mouse as an ergonomic switch - the mouse button switches being simply connected to the cord, no use being made of the internal electronics. A PIC 16F84 microcontroller was interfaced with an ISD 1016A "chipcorder" IC. This device, an addressable non-volatile audio record/playback chip, had my voice saying the alphabet recorded on it. The microcontroller then sequenced through the alphabet, and the user could hear this through the headphones. Letters were selected using the mouse buttons, and the created text was shown on an HD 44780 liquid crystal alphanumeric display, having two lines of 16 characters.

Here's a bit of documentation from 2001 -

The project stalled and was shelved. I'd contacted some people in the field of communications for the disabled, unfortunately they didn't like the fact that I wanted SWAN to be "freeware" - they wouldn't commit to further development and manufacture without sole ownership of the intellectual property rights. I got pissed off, especially since I was giving the design to them for nothing. End of SWAN...... Until 13 years later, I was given a smartphone, and the concept became do-able with minimal hardware cost, and a free "app".

The above single page description of SWAN is about all that remains of the project. The prototype went when my old shed biodegraded. The PICBASIC source code, compiled microcontroller hex code, the sound .wav file, schematic, bill of materials, costings, half-finished eagle PCB layout and other documentation all went the journey when my old 486's hard drive self destructed, many moons ago.....