Peril-Sensitive Sunglasses

That's a wrap!  We're done, so let's leave you with a brief video demo of the glasses reacting to sound and light.

This project details entry has some added information I wrote up after designing, building, and testing the prototype.

If you're interested in details about what worked, what didn't, and what was unexpected, give it a look.

No pictures this update - but some information on the process of making the whole thing work.

As seen in the last update, the prototype hardware construction is complete.  All the solder-y and nuts & bolts-y parts are done.  What's left is figuring out what exactly to do with all the data we are getting from the sensors.

This is easier said than done.  I can read the sensors just fine but deciding what to do with the data at a high level is a bit tricky.

Testing reveals that my current methods are good enough to tell the sensors are working -- but not well enough to reliably isolate anomalies. False positives (detecting Peril where there is none) and false negatives (failing to detect Peril) abound.  I need to re-evaluate my software.

It helps to define the problem.  Here is what we want:

Ignore slow and steady changes, and take action only on anomalous signals.

To do this we need to:

• Read sensors regularly and keep a history; from that we can establish what "normal" is.
• Figure out what "anomalous" readings actually means for each sensor. (This can be different per sensor - is slow but steady change upwards for the past two minutes anomalous for Ambient Temperature?  What about for Ambient Sound level or Ambient Light level?)
• Constantly re-adjust for one, and watch out for the other.

No matter what method or thresholds we use for sensors, the common thread is that we need to take and keep accurate measurements over time.  Only by comparing what's happening to what has already happened can we make judgments about normal vs anomalous.

The existing software does this, but as stated - not well enough.  Some software work should sort this out.  I'm reminded of why Signals Processing is an entirely separate discipline, because it's easier said than done!

It's said that the difference between a hack job and a professional-looking unit is measurement and planning. This is the first kind.

The electronics are now all packaged up into a handheld unit, and verified working.

This version is also a lot bigger than it needs to be, because when making just one of something and wiring up off-the-shelf things, size is never optimized.

Ideally the PPDD would be worn against the skin like a necklace, and powered by a compact lithium-polymer cell, but that's wishlist stuff.  Functional prototype comes first.

I used the Solarbotics S.A.F.E. acrylic enclosure, with a 4xAA battery holder "backpacked" onto it.

The two copper pads for measuring Galvanic Skin Response are placed such that it's natural for fingertips to touch them.

Sound Sensor mic is inside and the temperature sensor is just peeking out the enclosure.  There are a lot of "vent holes" in the enclosure so the sound sensor should be fine.

Next is actual testing and fine-tuning sensor response!

It's time to move the hardware off the breadboard and turn it into a functional prototype.

Because we're making only the one to play with and there's no time to do things like a custom PCB, we're going to need to use a protoboard, have wires everywhere, and generally wind up with an end result that is much larger than it needs to be.  But it will work the way it is intended!

All the electronics and sensors will be in a centralized control box, except for the ambient light sensor.  I am trimming some plastic from the glasses and fitting it into the glasses frame.

Here is the light sensor peeking out from a hole in the top of the frame.  

It makes sense that the ambient light sensor needs to be out in the open. (All other sensors can be squirreled away to some extent.)

The glasses will be connected via an umbilical to the main control/sensor box - the PPDD (the Potential Peril Decider/Detector)

Here is the prototype PPDD all wired up onto an Arduino shield protoboard.

Some wiring and testing needs to be completed first.  Hopefully I haven't screwed anything up in the process of moving from breadboard to protoboard.

The last sensor to work in is the Sound Detector Board.  I am using the ENVELOPE output, which is an analog representation of the volume of detected sound.

I haven't actually given it a proper run-through yet but I'm confident it will work just fine.

The ADC reads about 20-30 in normal quiet workshop conditions.

Clattering a tool near the sensor on the bench shoots it up to about 100+.  Loudly clattering the tool intentionally can easily spike 150+.

I don't see any reason the same treatment for other sensors won't work - regularly read values into a ring buffer and calculate a dynamic threshold to use based on the standard deviation of readings.  Slower, more gradual changes will be ignored but sudden changes will trigger the Peril-detection part of the code.

This is the last sensor to fiddle with; I think the sensor suite is complete now with:

• GSR (Galvanic Skin Response) sensor
• Sound Sensor
• Ambient Light Sensor
• Accelerometer (including freefall detection)

Peril will detected either reactively or pre-emptively based on the readings from these sensors.

The tiny light sensor board gives some nice and stable analog readings based on amount of ambient light.  It gives a "0" well ahead of full darkness however.

Software-wise I treat it almost identical to temperature sensor - read values into a circular buffer and calculate "standard deviation" of past readings, then compare that to the most recent reading.

It responds quickly to light changes so I can use past readings to generate a dynamic threshold - too high of a change in light value either high up OR down triggers the Peril-sensing part of the code.

It needed only a little extra tweaking - the math for generating dynamic thresholds, etc doesn't work well at very low values so if the readings are already very low (e.g. ADC reading is less than about 10) I just substitute some known static thresholds instead.

I think I'll try to mount this sensor inside the glasses frame.

I attempted to make a crude GSR sensor (Galvanic Skin Response) consisting mainly of a voltage divider, two copper plates, and an analog input.  

I used this wiring (source: http://www.instructables.com/id/Stress-Makes-Art-Galvanic-Skin-Response-and-Visual/ )

I can certainly get results from it and at first glance the data looks useful, but it's unclear whether I'm reading what I think I am.  I can certainly tell whether there is skin contact, at least.  But beyond that it's inconclusive.

I subjected myself to various stimuli (consciously calming my mind, consciously tensing up, jabbing my hand with a screwdriver, hyperventilating, looking at pictures of pretty girls, and things of that nature) with the contacts applied to my skin but was unable to find any useful patterns besides "skin contact yes/no".

More testing and study is needed if time permits.

Integrating the TMP36 temperature sensor was simple, I've used it before.  It looks like a transistor and outputs a voltage proportional to temperature.  Adafruit has a thorough guide on its use.

Making the sensor work was easy.  Using the temperature data to make meaningful decisions about detecting Peril is less easy.  Certainly, at a high level I want to assume Peril is present whenever the temperature rises or drops too suddenly, but how do you go about doing that, exactly? What temperature, exactly, does Peril drop in at? 

The devil is in the details, and when you are building something new it's all details.

I did choose a hard high and low temperature to use as "hard" Peril indicators (to detect, say, being engulfed in flames or ice) but I still needed to decide how to detect sudden, abnormal shifts.

I realized that a "sudden" shift in temperature is really a temperature change (up or down) that happens too quickly.  To measure temperature is easy - read the sensor.  But to decide whether the temperature change is too quick requires comparing temperature change X to an amount of time Y.

Change over Time is textbook calculus something.  It might definitely be a derivative, or maybe an integral.  Either way I don't know a lick more about it than that so I settled for a software framework that measures temperature regularly and creates rolling averages at different times.  A change of temperature in excess of X in time period Y is considered Perilous.   (X and Y values are a little up in the air right now but I'm sure a little fiddling will reveal something useful.)

That about wraps it up for the guts of the Temperature Sensor.

(Monitoring body temperature was out since it would require inserting a temperature probe somewhere into the body, which I'm fairly certain would be considered Perilous by other sensors, and the sunglasses would simply wind up being a self-fulfilling prophecy and possibly a paradox of some kind and that just wouldn't do at all.  So the temperature sensor remains more or less aimed at 'ambient'.)

TIP: For reasons that have to do with the output of the TMP36 sensor being high impedence, reading the sensor regularly and frequently and rapidly with the ADC can be glitchy.  A solution is twofold: 1) when you read the sensor, pause 10 milliseconds or so after each analogRead() of it, and 2) every time you want to read the sensor, read the sensor twice and throw away the first result.

Tested the Accelerometer for usefulness, and things look good!  

A basic software framework is taking shape as well.  

In this case, when too much movement at once (like a jerk or a jolt) in any direction is detected, the system considers that to be an indication of Peril.

The Arduino calculates a rolling average of each sensor (in this case, each of the 3 axes of the accelerometer).  The rolled average is used as the basis to calculate a dynamic threshold.

When the read value of any axis exceeds that axis' rolling average value by at least 50%, that is considered to be too jerky, too sudden, or otherwise abnormal.  The software then sets the Peril Detected flag.

The Accelerometer is a 3-axis analog output unit from Modern Devices which I had on hand: http://moderndevice.com/product/3-axis-accelerometer-module/.  The jumper visible is a simple hardware method of adjusting the sensitivity (6g - installed, or 1.5g - not installed)

This unit also provides a digital output if 0g (freefall) is detected.  This is automatically considered Perilous.