When I am working on a project, I like to build things that are the most cost efficient possible but still of good quality. So even those on a tight budget can afford to build their own. It's hard to give the exact amount of money spent, as many things are second hand or from the tip. Here is an approximation :

Electronic :

• Raspberry Pi (41$). I built my project with a Pi 2, but the B+ should also be fast enough.

• Teensy 3.X (20$). I love building projects with my Arduino UNO. But the specifications of this tiny Teensy board are really crazy! (32 bits cortex M4, 72MHz, 256K Flash, 64K RAM, 34 digital PIN...). I think it's very useful to keep one at home. If one of your projects needs more PIN, more memory, more speed or just needs to be smaller, you can easily switch as it's fully Arduino compatible! Just install the Arduino plug-in and keep developing! The only little thing you have to be careful of is the access to the button to reprogram. I had to wire an external button when the table was built and I still needed to calibrate and debug.

• The CIS sensor. You can easily get one from any old scanners which can be found everywhere. Or buy one on Ebay...

• LED IR (0.1$). I used some 950nm.

• The TV. It's hard to give a price, mine is a LG 32" that I bought second hand for 80$.

• Wire

• resistor

• Computer Fan. You can find some in most old computers, you don't need to buy one.

• Male and female headers

Some parts can be a bit difficult to solder, I recommend having a good soldering iron. At home I have been using a WS81 from Weller. A bit expensive to buy but it's precise and you can keep it for 20 years ;-)

The wooden box and table :

This part is gonna depend on your TV screen. This is for my 32" LG TV :

The TV box :

• 2 panels of MDF wood 10mm for the sides : 140 x 845mm
• 2 panels of MDF wood 10mm for the sides : 140 x 560mm
• 1 panel of MDF wood 10mm for the top : 950 x 650mm
• 1 panel of MDF wood 3mm above the top : 950 x 650mm
• 1 panel of Pine wood 18mm under the TV : 825 x 560mm
• 2 large Framing lumber supporting the TV : 37 x 54mm (300mm long)
• Small Framing lumber to ease the screw 15 x 20mm
• Screws

The Table under :

• 1 panel of MDF wood 18mm for the top of table : 670 x 350mm
• 1 panel of MDF wood 10mm for the layer under the table : 670 x 350mm
• A very large framing lumber for the table's feet 37 x 54mm, 280mm long(2 for each feet)
• Screws

Approximate cost of the wood 70$

I also used a Trotec laser cutting machine to engrave some characters on it and to cut several small parts of the wood. You can rent it for 25$ an hour from the Fablab next to my place. As I often help them on electronic projects I can sometime have it for free :-)

Touch detection :

• 1 Plexiglass 5mm for the support : 750 x 460mm
• 2 Plexiglass 2.5mm, 735 x 12mm
• 1 Plexiglass 2.5mm, 430 x 12mm (protecting the sensor)
• 1 Plexiglass 5mm, 430 x 12mm (with the LED on it)
• 8 mirrors 150 x 15mm (approximately as it's hand cut)
• To create a IR lighting paper filter I combined a 106 primary red paper filter with a 181 congo blue paper filter
• black masking tape

As Wikipedia is way better than me on explaining :

"Contact image sensors (CIS) are image sensors used in flatbed scanners almost in direct contact with the object to be scanned. Charge-coupled devices (CCDs), often used for this application, use mirrors to bounce light to a stationary sensor. CISs are much smaller than CCDs, use typically a tenth as much power, and are particularly suitable for low power and portable applications, often powered over USB.^[1]

A CIS typically consists of a linear array of detectors, covered by focusing lenses and flanked by red, green, and blue LEDs for illumination. The use of LEDs allows the CIS to be highly power efficient, allowing scanners to be powered through the minimal line voltage supplied via a USB connection. CIS devices typically produce lower image quality compared to CCD devices; in particular, the depth of field is greatly limited, which poses a problem for material that is not perfectly flat. However, a CIS contact sensor is smaller and lighter than a CCD line sensor, and allows all the necessary optical elements to be included in a compact module, thus helping to simplify the inner structure of the scanner. With a CIS contact sensor, the scanner can be portable, with a height of only around 30 mm. CIS is both a key component of, and widely used in, scanners (especially portable scanners), electrographs, bar code readers and optical identification technology."

Basically, the CIS you find in a scanner doesn't read lights in two dimensions, but on one line. With the one I tried, there are 2700 different light detectors that we can use as 2700 pixels on a X position.

As I couldn't find any tutorials explaining how they work on the internet, I tried to understand the logic by myself. I started by taking a printer scanner from the trash and opening it. Please be very careful with these devices, they use 220V! Don't do it if you are not sure of what you are doing. Always manipulate them when it's not plugged in and isolate any high power part.

By visually checking all the lines going from the control board to the CIS you can try to identify the 4 lines controlling the RGB LED, they are generally located at the extremity of the CIS board. With a multimeter, you can check where those lines are going on the communication flex.

Then you can check if some lines are bigger than others (it's probably going to be the power and the ground, you can verify which one it is with your oscilloscope when the power is ON). Plug an oscilloscope between the other lines and the ground to understand what they are doing.

I tried two CIS sensors from different brands and they where both using the same communication protocol but not on the same "line order on the flex". When I turned the scanner ON, I could see some data passing. It means that the device was probably testing the sensor when it is turned ON.

- One line was giving a perfect square signal, I deduced that it was a signal clock.

- One signal was turning ON and OFF at the beginning of the clock transmission, it's the Latch signal

- one signal was sending a lot of analog value. Those values changed when my hand was next to the sensor. It is the values from the CIS.

So I made a test program the on a Arduino UNO board. You can download it at the end with the project.

With this, you can read on the serial port 16 average values (I call them pixels) of 168 light sensor on the sensor, so almost the 2700 total. Just plug in the lines as explained at the beginning of the above file on any Arduino board and read the data on the serial monitor.

Last week, I was trying to find some solutions to improve the cover of the detection area. To do this, I wanted to have a second sensor to test new configurations and speed. So I went back to the tip and I took several printer/scanner from different brands.

I was a bit frustrating to discover that some scanner have a different kind of sensor. No need to open the entire device, they are pretty easy to recognize, as the black plastic package is way bigger (picture 1). Inside, they have a way more complex mirror architecture, a lens and... a very tiny sensor: A CCD! I will try to hack those one to see the possibilities soon, but for now, I needed another "normal" 22cm CIS. So I moved on!

Second printer I opened was from EP*** brand (Picture 2). The flex connection had 12 lines on 5V :

• 1 Analog OUTPUT to 4 are big lines easy to identify for the LED
• 2 (large) VCC? 5 (large) seem to be VCC
• 3 GND 6 GND
• 4 (thin) has a square signal with a 100ns period, the clock!
• 5 (thin) has a short square signal every 20us.
• 6 (thin) has a short square signal every 20us, another latch?? the latch.
• 7 (GND)
• 8 VCC
• 9-12 RGB LED

After few hours of test, I couldn't find the good protocol to have the OUTPUT response. Even when the CIS was plugged in the scanner, the analog wasn't giving any clear response (on startup test as the printer was then reporting an error).

So I moved on to a third printer from H* brand. The flex had 14 lines on 3.3V :

• 1 The analog
• 2 VCC
• 3 GND but not linked directly to the other GND
• 4 VCC but not linked directly to the other VCC
• 5 A latch every 40us
• 6 GND
• 7 ?
• 8 GND
• 9 CLK (clocking every 500ns)
• 10 VCC
• 11 - 14 RGB LED

I could access some PAD from the different line (picture 3). After a bit of testing I succeed to have a clean answer on the analog line (pictures and explanations 4 and 5).

The CIS I was using since the beginning is from CA*** brand. Here are the lines :

• 1 Analog
• 2-3 GND
• 4 VCC
• 5 ?
• 6 Latch
• 7 Clock
• 8 ?
• 9 - 12 RGB LED

Now we have a working CIS sensor with Arduino, we can start building our first touch screen:

You can take the glass from the scanner to use it as the surface. On one side you add the sensor and on the other side a IR LED. Power the LED (don't forget to add a resistor depending on the spec!) and start reading the sensor, you will see that a part of the serial data will raise. On a 10 bit ADC like the one on Arduino, the range value is 0 to 2^10=1024. On my test, my value is always between 550 to 980.

Two important things :

• Any other light can disturb this experience, especially sunlight! So close your curtains, don't point any light directly to the sensor and ideally, add some black tape around your sensor.

• Most LED have a very small emitting angle (around 20°). Don't be to surprised if only a part of your sensor sees the light, it just means that the LED is too close!

To visualize this I made a little processing program that you can download at the bottom of this with the project. This program shows 20 different cluster of the CIS sensor. If the average analog value of the zone is close to 1024, the cluster is white. If it is lower it gets darker. As Processing needs to communicate with "Serial.write" data and not "Serial.print" like the Arduino Serial port, you need to upload this new Arduino code on your board.

Now, if you put your finger between the LED and the sensor, you should see the cluster getting darker. Congratulations! You have made your first touch interface!! Now you can detect the area where your finger is, not very precise, but functional!

To improve the precision of the detection we are going to have to complexify the system by adding some LED on different digital ports of the board. As you can see on the pictures, I now have 15 LEDs (choose as many as you want) wired in a plastic brown sticks. The distance between each LED is 5cm so the total distance is 70cm.

Now, how are we going to detect a very precise position?

1. we need to check if the system is detecting something. The method is gonna be :

• Turn ON LED 1,

• Read the sensor and check if something is blocking the light

• If something is in the way store the pixels above calibration in a table

• Turn OFF LED1

• Turn ON LED 2

• Read the sensor and check if something is blocking the light

• if something is in the way store the pixels above calibration in a table

• Turn OFF LED2

• ...(do this for every LED)

Now we have stored all the areas detecting something.

2. calculate the function

I know school is often far away and mathematics didn't seem very useful at the time. But do you remember the basic function Y = AX + B ?

Here we are going to have a full application of this. We are going to calculate the function crossing the LED and the middle of the area the CIS sensor detected.

To simplify, we are going to set the XY origin of the graphic on the first LED called LED 1. We can then calculate B for every LED (in mm) :

LED 1 = 0, LED 2 = 50 (50mm distance), LED 3 = 100, LED 4 = 150...

The A is a bit more tricky, if Y = AX + B then A = (Y - B)/X

If we take the average of the pixels detected (in the program it's "(maxi + mini)/2") we can calculate the exact height (Y). We also need to transform the pixels in dimension.

(NB_PIXELS-(maxi+mini)/2)*CIS_HEIGHT)/NB_PIXELS+(SCREEN_HEIGHT/2)-(CIS_HEIGHT/2)

and as we have Y, B and X we calculate A :

A = (Y - B)/X

Now we have A and B of all the lines going from every LED to the middle of the area they can't see on the sensor. We have function1 : Y = A1 X + B1 and Y = A2 X + B2

We can then calculate the crossing of those functions :

The coordinate X and Y of the crossing will be :

X = (B2 - B1) / (A1 - A2)

Y = (((A1*X) + B1) + ((A2*X) + B2)) / 2

The X and Y are the crossing coordinates. If you calculate all the functions crossing point and you average them you should have a good approximation of the central position of the object.The multi-point can't work with only averaging, it need a more complex algorithm.

As we can see on the picture, this method is not really perfect. With 15 LEDs on one side, you can see on picture 1 there are still many areas we cannot detect.

So how are we going to make the detection area bigger?

• The easiest solution would be to add some LED which point to the sensors. This solution, which you can see on picture 2, would cover the entire central area but not the one closest to the LED. It's not that good as It would need a lot of LEDs, which would make our system go slower.

• The non detected areas are on the sides. An interesting solution could be to improve the distance between the LED and the sensor from the screen. This would solve part of the problem in the non-detected areas as you can see on picture 3. But the further the LEDs are from the sensors, the more interference we are going to have.

• After a bit of testing, I decided to add some reflective mirrors on the two sides. With this solution, if you add just a few LEDs which point to the side through the mirror you can cover a wide area. As you can see on picture 4.

• We could imagine a mixed solution like on the picture 5

• Another solution could be a combination of the three previous points. The sensors and LEDs are a bit far away from the screen, some mirrors cover most of the UP and DOWN sides. It's close to the picture 6. You can probably add some LED to cover the corners like on picture 5. Still not perfect, but almost entirely covered!

Using 2 CIS sensors

My new solution was to add a second CIS sensor (picture 7). Now, the entire screen is almost covered! But as two sensors is slowing down the system, I can play with the number of LED I use, and with the number of pixels. This way I am quiet flexible on the cover, the precision and the speed I want!

For example, if I need to emulate a keyboard for a Nintendo Nes, I only need 8 buttons. I can split my screen in 8 parts and detect if something is on this zone. Only 30 pixels (15 par sensor) and 8 LEDs is way enough to do that and I can reach around 40Hz speed.

If I want to emulate a mouse I need access to the entire screen with some precision. So I ll use 100 pixels (or more) and 16 LEDs. The speed will drop down to 15Hz but I can click where I want!

On the video, I show the Processing program running on the Raspberry Pi. I only used one CIS as it was before the 2 CIS upgrade, but the position still takes some time to synchronize. This problem is only because Processing is a heavy program, so the Raspberry Pi is struggling to refresh and synchronize the screen and the serial port fast. I had to add a delay on the Teensy program to slow down the communication. If you try the Processing GUI on a more powerful computer, it will be perfectly synchronize. 

As my LEDs don't have a big enough angle to cover the two sensors, I added a second LEDs in parallel pointing to another direction. I just had to adapt the resistor to have twice the power, and it's done!

As I was saying, the multi-point cannot be perfect using a light triangulation technology with a CIS sensor. It is pretty complex to work on this part... But it is also very fun to imagine many different solutions on how to have the best results without adding problems.

Here is how I made mine :

• First, I store all the value of the CIS in tab[][] for every LEDs

• Then I calculate all the factor A and B of affine functions Y = A * X + B going from every LED to the center of the "black zones" that the sensor don't see (in red on the picture).

• I selected the first function of the first LED (lets call it N) and calculated the crossing with the first function of the next LED (N+1). This is my first reference point. The very important step here is to remove those two functions by decrementing k in tab_position[LED][k][Black_zone] (the tab where the B and A are stored).

• Now I just check if the first function of every next LED (N+LED_LEFT) are going close to this point. If they are, I adapt the position of the reference point and remove them as well.

• When I checked all the other LEDs, I just start over for the second point. I can perfectly get the position of two LEDs or more (like on the second picture with 4 points) as long as several black zones detection for a LED aren't detecting two points in the same time. Their is still some improving work to do on how to use those "big black zone" detecting two points. In my code I choose to ignore them as I could.

This algorithm isn't perfect, but it allow me to use some multi-point for games, as I can choose where the detection points are (see the NES remote on step "Final program, let's play!"). I will keep improving it, and hope some people will advice and help me as well :-)

On the Fritzing picture, you can see how everything is connected.

A good thing about using a Teensy Board is you have 34 digital Input/Output. So we have enough free digital PIN to control 30 LEDs without the need of external LED drivers. As only one LED is on at a time, we don't need much power either.

In this project we are going to use the following Pin :

• RX/TX (digital Pin 0 - 1) for the serial communication

• CIS sensor control (digital Pin 2 - 3)

• CIS sensor read on analog (analog Pin 9)

• direct LED control (digital Pin 6 - 22)

I now have two LEDs instead of one for every PIN (to improve the angle for two sensors). The LEDs are powered on their Anode, which means you have to set your Pin Low to turn the LED ON. A 110ohm resistor is wired on the power line to avoid sending to much power on the LEDs.

I have already talked about the great hardware specifications of the Teensy board. But combined with the Arduino software environment, the Teensy also offers some great possibilities. But first let's install the plug-in

Now you should see a category "Teensy" when you choose your board. And if you select it, you have access to several other options like the "USB Serial Type" to instantly transform your board into a USB keyboard or mouse (some possibilities can be found on Arduino Leonardo). The USB communication type is a "Full Speed" which means it can communicate at 12Mb/s. You can also change the speed of the board (on this project I used the overclock at 92Mhz).

For now, all of my Arduino UNO codes have been successfully built on the Teensy which means the adaptation between the ARM Cortex M4 and the AVR Atmega328 have been very well made. I tried to write the code to be understandable. Here is the structure of the main loop :

• A for loop from 0 to the number of LED

- Turn an LED on (as the LEDs are wired with the Anode on VCC and Cathode on the PIN, you have to give a LOW state to turn it ON)

- ClearCIS(), The sensor has to be cleared once before reading the value... I don't know why. If anyone has an idea of how I could speed up this part, I would very much appreciate it.

- readCIS(), read the sensor, if a value is above the calibration it is stored and the function return 1. If something have been detected, the flag_detect is set to 1.

- Turn the LED off (HIGH state for OFF in this configuration)

• If something has been detected, then the flag_detect is set to 1 and you enter the IF function. You can then choose your calibration between the following :

- OPTION 1; send Serial monitor.

Basic debugging : This option allows you to read the 16 clusters directly on the serial monitor.

- OPTION 2; calculate position + send Serial monitor

Debugging position : send the coordinates X and Y to the serial monitor

- OPTION 3; send Processing interface

Debugging visual : look at the CIS results on a Processing GUI. The interface is going to have to be adapted to the number of LED (16 default).

- OPTION 4; calculate position + send Python Pi interface

Send the position to the PyQT interface

- OPTION 5; calculate position + send mouse (only on Teensy)

Send the position as a USB mouse interface. To use this option you will have to go to "Tools" -> "USB Type" -> "Keyboard + Mouse + Joystick". (See the Warning at the bottom of this Step)

- OPTION 6; calculate position + send Keyboard (only on Teensy)

Send the position as a USB keyboard interface. To use this option you will have to go to "Tools" -> "USB Type" -> "Keyboard + Mouse + Joystick".

- OPTION 7; Point and click. Same as mouse but click instead of following the position

• You can eventually add a delay at the end of the loop(). Depending on the speed you need

WARNING: OPTION 5 uses the command mouse.mouveTo(X, Y) which doesn't work on a Linux station. If you want to do the same configuration as me with a Linux Raspberry Pi you are going to need to use the UGLY HACK made by Paul Stoffregen who seems to be one of the developers of the Teensy Project. He is really good and he helps a lot with debugging if you have any questions. Thank you Paul!

For this hack, you need to modify a part of the file name usb_desc.c It should be installed in your Arduino folder. The new configuration given by Paul is slightly different but he makes it work! By then, the mouse.move() function is not working anymore.

On the Raspberry Pi side, there are many different ways to show some data with a GUI. I choose to write a basic interface with PyQT following a friend's advice. Now, the interface has a full screen picture, read the serial datas and write them on a little panel on the left of the screen and draws a cursor following the serial position received.

The program run on Python3.2 and Pyqt4 on the Raspberry Pi. You can easily find many tutorial to install them and run a program.

As Raspberry Pi had made a good release on installing and running Processing on the board. I also decided to adapt my debugging code to show it on the big screen, it's very slow but but great for debugging and improving the Teensy code. To do this, you will have to rotate the screen 90° (If your sensors and LEDs are on both sides like me).

• To do this, you will have to modify the startup file config.txt. Here is how you do it :

• Open the Terminal (or use the one on startup)

• Access to the file : sudo nano /boot/config.txt

• Then go at the bottom and add : display_rotate=1

• Then : ctrl X

• And y to say yes on saving the program.

• Restart you Raspberry Pi : sudo reboot

Now your screen is rotated, you can play with the number to have different rotation (2 is 180°, 3 is 270°, 0x10000 for horizontal flip 0x20000 for vertical flip)