My 7yo son has Stage 3 Autism Spectrum Disorder. He his non-verbal and is learning to use the PECS (Picture Exchange Communication System) in order to communicate his needs and wants with myself and his family.
He has certain shows and music he likes to watch on his TV, and if something comes on that he doesn't like he can get very frustrated, and can even have a meltdown.
Also, being a 7yo, we like to control what he watches, and when he watches it. So it came to mind that I would be able to roll all of his needs, and our control into a single package that will help him with his independence.
I have thought of a Remote Control device, based around the PECS system. Essentially it's a board that displays images corresponding to the shows he has access to, and when he touches the image on the board it turns his TV on remotely via IR if it's off, and then starts to play the corresponding show or YouTube playlist on the Chromecast attached to his TV.
I have been working on the software stack for the platform as my time allows, and I have so far developed a functional base API project in ASP.NET Core 2.2, and a plain HTML/CSS/Javascript based admin website that leverages the API.
I have login functionality working, the main interface, software update reporting and applying, and I am in the process of making the setup interface work. I also have the code in to scan for, connect to, and play media to the Chromecast (or any other device that uses the Chromecast Cast system).
Since the Raspberry Pi 4 was released during the development, I have ordered a Raspberry Pi 4 1GB RAM model (1GB should be plenty for this task) and a 240GB SSD with USB3 external enclosure.
This will allow me to implement reliable transcoding and streaming of the video to the Chromecast rather than just pushing URLs.
Hopefully today I will be doing some initial testing of the code to get media playing on the Chromecast by hosting the ASP.NET Core 2.2 API project on my Windows Developement laptop.
I have ordered some hardware to start building my prototype, and begun developing the software stack to run the platform.
Since I am a C#/.NET guy mainly, I have gone with what I know and am developing a service that runs ASP.NET Core 2.2 to both serve up the API and to actually do the work making things function.
The administrative interface used to control the platform is a plain HTML/JS SPA package that runs on the client and communicates with the API.
It is my plan to turn both the SPA and API into debian packages that I can update via an APT repository.
The hardware I have settled on so far is a Raspberry Pi 3 for my initial prototyping, with the option of switching to a Pi Zero W for a production run if I get that far. The reasons I am going with the Raspberry Pi are:
Well supported platform that receives updates to it's OS stack.
Has kernel framebuffer support for the chosen LCD modules.
On the linux software front I will be looking at doing the following:
Serving a friendly name to connect to the device via mDNS
On initial powerup or after a "Factory Reset" running an Open WiFi network with Captive Portal to do initial device setup (Setting host name, WiFi network settings, Changing admin credentials, etc)
Checking for updates to the platform via apt.
Read-Only root filesystem, only allowing writes during a system update process.
Configuration and database on a second writable filesystem.
I have begun working on the basic software platform for the project while I am in the process of researching and acquiring the hardware to start building the device.
The device needs to be robust. Due to my son's condition he can be incredibly rough, and sometimes violent when he has a meltdown. If he has a meltdown and lashes out, he needs to not only NOT hurt himself on the device but the device needs to be able to withstand the punishment.
Easy To Use
The PECS system is very simple. Images represent the needs of the user, and communication takes place by simple selection of an image. The device needs to include this incredibly simple interface.
Easy To Administer
From the perspective of us parents, we need to be easily able to add/remove/update the choices available on the device.
Low Power
Keeping the power usage down is a definite plus. Since most of the time the device will just be waiting for an image to be selected, properly implementing low-power modes in the hardware is a definite design goal.
Secure
Since the device will be responsible for controlling what my son watches, the device needs to be built securely so that it presents as little an attack surface as possible for hackers. The last thing we want is the list of shows/videos that can be shown to be replaced with content that is not suitable, or even illegal.
Initial Thoughts
My initial thoughts on the hardware for the device are a Raspberry Pi 3 connecting to my home network via WiFi. This way, I can leverage the security features of Linux, the ease of developing the software stack to control the device, and the hardware support for serial device protocols (I2C, SPI, etc).
For the front end interface, I am thinking 6 small 2.4" 320x240 TFT display modules connected to the Pi via SPI to display the PECS images. To perform the actual selection sensing, I am thinking of using Capacitive Touch Sensors made from a thin layer of clear acrylic, then a layer of Indium Tin Oxide coated plastic film, then a thick layer of clear acrylic behind it for strength, then the TFT mounted behind that.
To implement the low-power states, if the board hasn't been used for some time, the TFT displays can be powered down and the backlights turned off under software control. The capacitive touch sensors or even an accelerometer could sense the device being touched, and power the displays back on.