It's a Testing Platform

The goal of this little project is to have a basic general purpose platform for trying out key-to-symbol mapping schemes.

Last night I desoldered some keyswitches from a junked keyboard, took some measurements of my hand on cardboard, threw together a laser-cut design in SolidWorks, cut the design out of 5 mm plywood, hot-glued the pieces together, popped in the keys, soldered together a messy per-key pull-down circuit on a perfboard, plugged the mess into an Arduino, wrote some firmware to send 2 bytes with key state every time the state changes (debounced), and wrote a basic Processing sketch that uses java.awt.Robot to send keypresses to my OS. With those basics in place I can start playing with different ways to translate button presses into characters or control keys.

First Test Subject: Arpeggio Input

In my last project log I laid out problems that I see with the input model used by chording keyboards. I am collecting evidence that shows chording keyboards to be inherently slower than their traditional counterparts, due to greater mental and physical effort when executing them. I believe that a solution is to put arpeggios in the place of chords.

Not a Chorder, an Arpeggiator

An arpeggio is a "broken chord" where the notes of a chord are played one-after-another instead of together. In keyboard terms, to produce a character like '~' on an arpeggiating keyboard (term I just made up) you might press key 1, then key 2, and lastly key 4, where on a chording keyboard you would press keys 1, 2, and 4 at the same time. Even though common sense would dictate the former to be slower than the latter, because the time taken to press all the keys at once would probably take the same time as pressing just one key in the sequence, a closer inspection makes the situation less clear. It can be quite difficult to coordinate multiple fingers at the same time. Anybody who has learned to play chords on a guitar or even a piano knows this. With practice it can be done, but it is still slower than carrying out an action with one finger. In the hand and arm, the machinery of the fingers is physically linked. Try moving each of your fingers independently. You will notice that it is impossible to keep all other fingers still while moving one. I think this hints that more strength, focus, and muscle memory is required when executing chords.

Pressing buttons one at a time can be done extremely quickly, as we know from using traditional keyboards. Interestingly, I see anecdotal evidence that speed scales when decreasing the number of keys and the distance between them. This video of an extremely fast calculator user, and my own experiences programming on calculators in the past, inspired in me the idea that this might be the case. My hypothesis is that a keyboard which assigns one key to each finger can require multiple independent key-presses per character and still be as fast as a traditional keyboard. The fingertips do not need to translate through space, so the basic time unit is that required for a finger to depress a key. It seems quite possible that in less time than it would take to remember and execute a chord a user could remember and execute an arpeggio.

I do not yet have any data to back up my hypothesis, but that is why I built the Prototyper.

The Mental Difference

I had a eureka moment when I thought about exploring arpeggios for Keychange. Since the inspiration came in part from my experience programming for many hours per day on my trusty TI-83+ calculator, I pulled my old calculator out of storage and programmed on it a small prototype of the idea. I made use of 4 keys and decided on using the basic lower-case alphabet. Organized into a tree where each node leads to 4 others (one branch per button) all of the leaves (letters) could be reached in 3 key-presses. I set about learning to type "hello". Surprisingly, it only took a few minutes. Learning to type any letter took half an hour. After that I felt like the idea needed a closer look.

My experience with this prototype suggests that a powerful advantage of arpeggios over chords is that they are much easier to transfer to muscle memory. Instead of looking at some huge list mapping chords to symbols, arpeggios navigate you through a tree. You "hone in" on your target, which makes it much quicker to learn. After you have hit the key sequence for "a" a few times your fingers start remembering the correct order on their own. I believe that for some as-yet-unknown reason, based purely on my own experiences chording and with this prototype, it is easier to learn arpeggios than chords. Hopefully using the Prototype hardware I will be able to collect data to properly support or reject this hypothesis.

Flexibility in Organization

A big benefit that I see in arpeggios over chords is based in their usage in navigating a tree of symbols. I am currently taking a class that teaches about compression, so the trees used in Huffman coding were fresh in my mind when considering the idea. Using the concepts like Huffman coding borrowed from compression it should be relatively easy to take some representative sample text, run it through a computer program, and output an optimized symbol tree where the most used symbols are the ones that take the shortest amount of time to type. Last night I started playing with this by writing a short Python program to digest the c files from the Linux source-code and spit out a tree that would be optimal if I were Linus Torvalds.

In the process it occurred to me that there is a need for real-world data to put into the tree-generation process. I want to use Prototyper to empirically measure the time it takes to type out any sequence of two symbols, because I suspect that it will be much faster to type sequences that "ping-pong" between both hands. One hand can be preparing to press while the other is pressing. And there may be other real world effects to exploit that I don't see. Once I have the data I can run it through some off-the-shelf machine learning algorithms and spit out a tree that I can be confident optimizes for the actual physical system.

From there it is just a matter of getting data from more people and choosing a few good text corpora that represent programming and english. Then I can generate two authoritative symbol trees that will serve as the default "keyboard layouts" for Keychange. People will of course be able to create their own custom trees, but it is important to have standards for the average user to learn.

P.S.

1. I say arpeggio, but it is not perfectly correct because in my usage the order of the individual keys matters. In musical arpeggios the order is rarely specified, and it is usually just assumed to be in increasing order. I hope that the difference is not too severe to lead to confusion.
2. Arpeggios have a cool history in chiptunes.