Shorty's purpose is to be a platform for developing the firmware that will translate button presses into keypresses on a PC. I did not start working on it with the intention giving it a life beyond dev work on Keychange, but it has turned out to be so useful to me in general that I might develop it further into a stand-alone project. The killer use is supplying tactile input while using tablets.

My primary mobile computer is an MS Surface Pro. It does not have a built-in keyboard, and the floppy "cover" keyboards are terrible, in my opinion. You have to use a kickstand to hold the screen up when typing on them, which only has one angle (too vertical for comfortable use) and is really uncomfortable to use on your lap. So I use a super cheap netbook for typing work like programming. But when I design PCBs I love using Eagle on my Surface, because manipulating eagle with a pen is so much more pleasant than using a mouse. But if you have ever used Eagle, you know that there are lots of (weird / unintuitive but) useful keyboard shortcuts and using them is essential to efficient PCB work. Before Shorty I had to waste half the screen on the software keyboard and put up with the annoyance of it being slower to manipulate than a hardware keyboard.

With Shorty I can stop using the software keyboard. I threw together a super simple firmware that just has the 8 most common Eagle shortcuts mapped to the buttons. I stick Shorty to the back of my Surface where my left hand goes. Then I have the pen in my right hand. And altogether it makes a super powered PCB design tool. It is a really specific use-case, but such a helpful one for me that it will make all the extra work of fleshing out Shorty worthwhile.

Working out the lesson I learned from my previous grip prototypes, that comfortable typing will only be possible with rigid support, I spent a little bit of time mocking up a grip design that braces against the forearm. It is bulky, but it feels much more comfortable than the palm-braced designs I have been making so far. I might make a second version that is lighter, adjustable, and foldable to see if it could be a practical approach for the final design.

The PCB that I designed and ordered from OSH Park arrived. The design is in the Github repository for the project. I have assembled it, applied a minor fix (forgot to route traces for the programming header), and tested it with some basic firmware.

The design uses an RN-42 for Bluetooth communications. It is what lets Keychange emulate a Bluetooth keyboard and mouse. On the other side of the board is an NRF24L01+ 2.4 GHz transceiver module. It will be used for communication between two Keychange devices so that one can be used per hand without wires connecting them. In the middle is the Atmega32 that runs the show. To the side are connections for the buttons (there are extra connections for prototyping), for the joystick, and for the battery module (a LiPower boost converter).

The ergonomics of Keychange have been getting a lot of my attention. It is very important to me that this device is comfortable to use, because if it does its job successfully then it will be in my hands all day. If I get the shape and mechanics of it wrong then I will have a device that is unpleasant to use and possibly even one that could lead to injury (repetitive strain injuries are no joke for people who need to interact with computers all day). So I have been spending about as much time working on giving it a pleasant shape as working on the electronics and programming components that will make it actually work. Key to the effort are physical prototypes that I can hold in my hands.

Playdough Grips

For a while I was sketching out designs on paper and making mock-ups in 3D modelling software. This is usually how I approach the initial design of physical objects. But the sketches were difficult to produce because the shapes I wanted were organic and 3D. It was hard to convey the full shape with just lines in pencil or pen. 3D modelling helped with that problem, but it took much more time and could not let me keep up with the ideas I wanted to play around with.

So I quickly decided to try doing the modelling by hand. Clay would probably be the best material for the task, but I did not want to make the trip to the art store or spend a lot to order it online. I remembered from 1st grade that it is really easy and cheap to make playdough. Basically all you need is flour, salt, and oil. I already had all those things, so I made a big batch and spent a while making different comfortable shapes in my hand. My playdough was not nearly as good at holding specific shapes as clay would have been, but it did a great job of giving me intuition for what good shapes might be. I tried baking the prototypes shapes I made in the oven to harden them, but bizarrely the insides of them turned liquid and oozed out, leaving only a thin shell behind. They were still useful, but not very resilient.

I liked one of them in particular because it was small and especially comfy to hold. I took some Cherry MX switches and pushed them into little square holes that I cut into the playdough shell. It looked pretty weird, but it gave me some more ideas about how it might feel to press buttons while holding a shape like that. Unfortunately I learned that it is really fatiguing to press buttons on a shape where all of the support is coming from the base of the palm. The fatigue comes from the effort required to hold the thing still while pulling a finger down. My take-away was that the final design should have really good bracing against the wrist or even part of the arm. Just bracing against the hand will lead to wiggle room that makes typing feel bad and leads to more energy being expended per key press.

Laser Cut Grips

After messing around with playdough I decided that it would be useful to do prototyping in a material strong enough to allow me to mount buttons and electronics and have a working mock-up. So I turned to laser-cut 5 mm plywood. Working in 2D was much faster than working in 3D, but was not as flexible and the designs that resulted were not really comfortable. I explored two designs in plywood: a palm-braced minimal grip and a gun-style grip.

I look at both of them as failures because they are not that comfortable to hold. And the gun-grip looks too much like a gun. It probably would be bad to have that kind of association tied to the final design. My take-away lesson is that hand-made fully 3D prototypes are the best when exploring ergonomics.

The process of finding an optimal shape for Keychange has just begun, and many more physical prototypes are ahead.

Yesterday I finished the Prototyper hardware and wrote software to collect information on timing between keypresses. The firmware I wrote accepts commands over serial that specify a first key and a second key. Then it sends back the number of milliseconds between when the first key is pressed and when the second is. A Processing sketch running on a PC prompts the human being tested to press specified randomly chosen key sequences. It records the data collected from the hardware to a CSV file.

Data

After finishing the firmware and PC software and working out a couple of bugs I did a test run on myself. I recorded about 1000 key sequences (~2000 key-presses). Then I wrote a Python program to generate the matrix graph below, where average timing for every key sequence is normalized and color coded. The left-hand vertical axis is the "from" key, and the horizontal axis is the "to" key. The numbering starts with 0 at my right hand pinky and goes to 9 at my left hand pinky.

Observations

The data is noisy and I suspect that there is not yet enough of it to draw strong conclusions. But some preliminary observations can be made:

• The yellow spattering in the upper right seems to indicate that it generally takes me longer to go from my right hand to my left hand.
• My left pointer finger is pretty fast overall, which is strange because I am right handed.
• My right pointer finger is slow if the next key is on the left hand, but generally fast otherwise.
• Pinkies are the overall fastest. I think this has to do with the weird way I learned to type on QWERTY keyboards.

Improvements

Having data is exciting, but I need to make sure it really tells me what I need to know before relying on it. There are a couple of problems I need to resolve before I use Prototyper to collect much more data:

• While collecting those thousand samples I noticed that it took me a lot longer to get ready for certain key sequences. Because I was only recording the time between the first press and the second, that setup time was lost. For the future I will collect that time on the PC side and record it.
• The data gathering PC application indicates key sequences with colors. Green is the first button to press and red is the second. Often I would simply make the mistake of messing up the order of presses. This reflects a poorly designed GUI, not problematic sequences, so I'm going to replace colors with an arrow that pulses in the right direction.
• Just taking the mean of times for a key sequence when analyzing the data is probably a bad idea because a single outlier can mess up the whole set for a key sequence. I don't know much about statistics, but friends of mine do, so I'll get their help in future analysis.
• I should be recording when completely incorrect keys are pressed, because that is also indicative of a problematic sequence.

It's a Testing Platform

The goal of this little project is to have a basic general purpose platform for trying out key-to-symbol mapping schemes.

Last night I desoldered some keyswitches from a junked keyboard, took some measurements of my hand on cardboard, threw together a laser-cut design in SolidWorks, cut the design out of 5 mm plywood, hot-glued the pieces together, popped in the keys, soldered together a messy per-key pull-down circuit on a perfboard, plugged the mess into an Arduino, wrote some firmware to send 2 bytes with key state every time the state changes (debounced), and wrote a basic Processing sketch that uses java.awt.Robot to send keypresses to my OS. With those basics in place I can start playing with different ways to translate button presses into characters or control keys.

First Test Subject: Arpeggio Input

In my last project log I laid out problems that I see with the input model used by chording keyboards. I am collecting evidence that shows chording keyboards to be inherently slower than their traditional counterparts, due to greater mental and physical effort when executing them. I believe that a solution is to put arpeggios in the place of chords.

Not a Chorder, an Arpeggiator

An arpeggio is a "broken chord" where the notes of a chord are played one-after-another instead of together. In keyboard terms, to produce a character like '~' on an arpeggiating keyboard (term I just made up) you might press key 1, then key 2, and lastly key 4, where on a chording keyboard you would press keys 1, 2, and 4 at the same time. Even though common sense would dictate the former to be slower than the latter, because the time taken to press all the keys at once would probably take the same time as pressing just one key in the sequence, a closer inspection makes the situation less clear. It can be quite difficult to coordinate multiple fingers at the same time. Anybody who has learned to play chords on a guitar or even a piano knows this. With practice it can be done, but it is still slower than carrying out an action with one finger. In the hand and arm, the machinery of the fingers is physically linked. Try moving each of your fingers independently. You will notice that it is impossible to keep all other fingers still while moving one. I think this hints that more strength, focus, and muscle memory is required when executing chords.

Pressing buttons one at a time can be done extremely quickly, as we know from using traditional keyboards. Interestingly, I see anecdotal evidence that speed scales when decreasing the number of keys and the distance between them. This video of an extremely fast calculator user, and my own experiences programming on calculators in the past, inspired in me the idea that this might be the case. My hypothesis is that a keyboard which assigns one key to each finger can require multiple independent key-presses per character and still be as fast as a traditional keyboard. The fingertips do not need to translate through space, so the basic time unit is that required for a finger to depress a key. It seems quite possible that in less time than it would take to remember and execute a chord a user could remember and execute an arpeggio.

I do not yet have any data to back up my hypothesis, but that is why I built the Prototyper.

The Mental Difference

I had a eureka moment when I thought about exploring arpeggios for Keychange. Since the inspiration came in part from my experience programming for many hours per day on my trusty TI-83+ calculator, I pulled my old calculator out of storage and programmed on it a small prototype of the idea. I made use of 4 keys and decided on using the basic lower-case alphabet. Organized into a tree where each node leads to 4 others (one branch per button) all of the leaves (letters) could be reached in 3 key-presses. I set about learning to type "hello". Surprisingly, it only took a few minutes. Learning to type any letter took half an hour. After that I felt like the idea needed a closer look.

My experience with this prototype suggests that a powerful advantage of arpeggios over chords is that they are much easier to transfer to muscle memory. Instead of looking at some huge list mapping chords to symbols, arpeggios navigate you through a tree. You "hone in" on your target, which makes it much quicker to learn. After you have hit the key sequence for "a" a few times your fingers start remembering the correct order on their own. I believe that for some as-yet-unknown reason, based purely on my own experiences chording and with this prototype, it is easier to learn arpeggios than chords. Hopefully using the Prototype hardware I will be able to collect data to properly support or reject this hypothesis.

Flexibility in Organization

A big benefit that I see in arpeggios over chords is based in their usage in navigating a tree of symbols. I am currently taking a class that teaches about compression, so the trees used in Huffman coding were fresh in my mind when considering the idea. Using the concepts like Huffman coding borrowed from compression it should be relatively easy to take some representative sample text, run it through a computer program, and output an optimized symbol tree where the most used symbols are the ones that take the shortest amount of time to type. Last night I started playing with this by writing a short Python program to digest the c files from the Linux source-code and spit out a tree that would be optimal if I were Linus Torvalds.

In the process it occurred to me that there is a need for real-world data to put into the tree-generation process. I want to use Prototyper to empirically measure the time it takes to type out any sequence of two symbols, because I suspect that it will be much faster to type sequences that "ping-pong" between both hands. One hand can be preparing to press while the other is pressing. And there may be other real world effects to exploit that I don't see. Once I have the data I can run it through some off-the-shelf machine learning algorithms and spit out a tree that I can be confident optimizes for the actual physical system.

From there it is just a matter of getting data from more people and choosing a few good text corpora that represent programming and english. Then I can generate two authoritative symbol trees that will serve as the default "keyboard layouts" for Keychange. People will of course be able to create their own custom trees, but it is important to have standards for the average user to learn.

P.S.

1. I say arpeggio, but it is not perfectly correct because in my usage the order of the individual keys matters. In musical arpeggios the order is rarely specified, and it is usually just assumed to be in increasing order. I hope that the difference is not too severe to lead to confusion.
2. Arpeggios have a cool history in chiptunes.

I'm still thinking hard about the basic problems inherent to the success of an input device like Keychange. I believe that the combinations of solutions that past alternative keyboard projects used is mainly what relegated them to the historical dust bins. In this update I'm going to lay out what I think those problems are and how I intend to solve or avoid them.

Existing Chording Keyboards Are:

1. Unfamiliar - Looking at most chording keyboards, a non-techy person might have difficulty in even recognizing that the device is a keyboard. Each appears as an organically shaped high technology object covered in buttons. The gut reaction is fear and ridicule. Fear like fear of math. A person looks at some formulation and are told it is very useful, but the symbols used to describe it give an air of great complexity. People fear the unknown, so they avoid the device in favor of an easier alternative. Ridicule because such devices will be seen as tools for nerds who must care too much about the pointless, boring problem of digital text input.
2. Memory-Heavy - The basic premise of a chording keyboard is found in its chords. For every symbol there is a button combination that summons it. Unfortunately, the inherent trade-off is that a much higher mental effort is needed to get the benefits of the device. Most people will get discouraged when they find out they need to memorize dozens of chords before they can take advantage of a device. The traditional keyboard also requires practice, but a person can start using it immediately because every button is labeled with what it will do. For the traditional keyboard, efficiency is proportional to time. For a chording keyboard efficiency is zero for some time, and then it jumps up and slowly increases thereafter. The difference is a crucial friction point that prevents the wider adoption of chording keyboards.
3. Slow - For the bright, nerdy people who have traditionally used chording keyboards this is likely the most severe problem. Chording keyboards are inherently slower than their many-keyed counterparts. In reviewing the experiences of many chording typists I have found that it is not simply a matter of practice. Like in playing traditional musical instruments, motions involving individual fingers are much faster than motions involving the coordination of multiple fingers. Part of this problem is mental - it takes longer to order around several fingers compared to one. And part of the problem is physical - the machinery of our fingers is not independent per digit. Some fingers move together, and that makes many chords inherently more difficult to execute.
4. Bespoke - With unfamiliarity, this is a problem not inherent to the concept of chording keyboards. Alternative keyboards of all types have never seen wide popularity. So people who want them are forced to reinvent the wheel and come up with a custom solution. For those people who actually go and do it this is not actually a bad thing, I think, because they are more likely to be the type who would want a custom solution anyway. But when unlike people go looking for alternative keyboards they mostly just find those custom projects and face a wall. They do not have the skills to reproduce one of the projects, do not want to take the time, do not see a project that fulfills their needs, or cannot find enough details about any attractive project to reproduce it.

How Keychange is Different, and Why it will Win

Problem: Unfamiliar

Familiarity is a matter of context. Traditionally alternative keyboards have been fighting for the same space as the incumbent typewriter-style keyboard. Juxtaposing the two makes the alternative look like a loser. Most people are not going to bet on an underdog in a situation where they are mostly happy with the status quo.

Keychange sidesteps this issue of juxtaposition through virtual reality. In the context of virtual reality it is actually conceivable that the incumbent's advantageous position has been weakened and an underdog could be attractive enough to win. Keychange does aim to succeed as yet another alternative keyboard, but that is not the domain where it will choose to fight. As an input platform for virtual reality it can have the conceptual distance it needs to appear a daring underdog, rather than an unknown and scary threat to the existing paradigm. After winning there it might have the mind-share to take on the traditional keyboard, but that is a question to settle later.

Problem: Memory-Heavy & Slow

In solving this problem I think that Keychange can set itself apart from other similar projects. You may notice that this post sometimes uses the term "alternative keyboard" in place of "chording keyboard." The reason for that is I want to transition away from this being a project aiming to build a chording keyboard. I believe that many of the problems chording keyboards face are inherent to their model, and only by being willing to step outside their definition will this project have a chance at general success.

A requirement for great memorization effort before any use of Keychange is not acceptable for a device hoping to be used by ordinary people, or at least by virtual reality enthusiasts who don't actually care that much about keyboards and just want a good solution for VR text input.

Trying to make due within the chording model results in unacceptable trade-offs. To ease memorization a design can make the chord mapping follow some obvious structure, increase the number of keys so mappings are easier to follow, or just reduce the symbol-space (reducing the total number of chords). None of these solutions are very good. Coming up with an obvious structure is hard. Some commercial projects I found tried to achieve this by connecting chords to pictures involving the keys (see the microwriter). But the scheme is awkward at best. Increasing the number of keys defeats the purpose of the chording keyboard, makes it more expensive, and mechanically complex. Reducing the symbol space has been done by many commercial chording keyboard projects as well, but the result is more a toy that can be used for basic notes rather than a general purpose solution.

Minimizing Chording

In light of these problems, I am investigating models that are entirely separate from chording or hybridize with it. The most promising model that I am experimenting with eschews chords in favor of a sequence of keys per character. It is a powerful approach that I will explain with more depth in a post on an experimental scheme-testing keyboard that I am currently working on. A short example of the approach is: instead of pressing keys 1, 2, and 4 at the same time to get the character '~' you would press 1, then 2, and then 4. It might seem bizarre and even counterproductive to serialize what originally distinguished itself through parallel input, but I suspect that counterintuitively it will actually result in less faster learning time and greater typing speed. More information to come in a post on my "Prototyper" hardware, which I intend to use to test my hypotheses on the model.

Problem: Bespoke

It is relatively straightforward for Keychange to solve this problem. If it solves a problem people care about in an accessible way then it can become popular. If it becomes popular then it can set the standard for its domain. If it sets the standard then people looking to innovate further will have an attractive place to start. All of that should come naturally if I make it my business to enable each stage.

I argued in my first project log that Keychange can solve a problem people care about. To be accessible for further innovation I believe that it must be thoroughly open. I will make an effort to release as much of the work going into the project as open source, and keep it clean so that people can more easily take advantage of it. Also towards the goal of enabling innovation I will be putting a significant amount of effort into ensuring that the design is modular and can be easily modified for tangential purposes. The basic ways to accomplish that include breaking out unused pins on the PCBs, using well-known components, explaining why design decisions were made, clearly commenting the code, conforming to popular standards, and etc. I cannot write down a full list of these things. Instead I promise that I will give them careful thought during development.

P.S.

Oculus Rift

I ordered a Developer Kit 2 from Oculus Rift! Unfortunately it likely will not arrive until the end of prototype development in January. Fortunately, customization for virtual reality can wait until then. It is easy enough to simulate the relevant problems by closing my eyes and seeing how well I can type, or by trying to use a Leap Motion, Kinect, Wiimote, phone, or etc. for interaction while holding Keychange devices.

Better Information

I'm making two new sources of project information available:

1. The project notes that I am continuously adding to while working on the project. It is a very rough scratchpad of ideas and gives a general sense of what I am thinking about and where I am headed. Usually updates at least once per day.
2. The project now has a Github repository. Currently it just contains code for the experiments I am carrying out, but eventually it will hold everything needed to build your own Keychange, including hardware designs, a bill of materials, firmware, etc.

In the four months since initially putting up my "Chord Keyboard" project on Hackaday.io I've not done any physical work on it. Partially that's because of other projects and schoolwork, but mostly it's because I just haven't prioritized it. Chord keyboards have been built over and over since the dawn of computing (and even before then) but they've never caught on because the standard keyboards work well enough for most people. Every time I'd consider working on the project, I'd decide that my time was better spent working on things that lots of other people would find utility in.

But recently I was thinking about the project and had a breakthrough. I was trying to imagine an argument with some other person where my aim is to convince them to use my chording keyboard. The benefits that I see in the device were just not good enough to convince the other person to go through the effort of using my device over the existing standard solutions.

So my mind wandered to other contexts where the existing solutions might be worse off. And that's when I started considering chording keyboards in relation to virtual reality. It's an area of development that I've been following closely after trying out the Oculus Rift last year. In the time since that (really cool!) experience I have been exposed to lots of awesome virtual reality demos and the relayed experiences of friends who have interned at Oculus or played with modern virtual reality systems.

Taking all of that together, I am convinced that virtual reality is going to be a big deal sometime in the near future. And there's a little niche for me to carve out within it by developing a really good keyboard for virtual reality. If I succeed, then a much larger group of people will benefit from my work compared to if I had simply added another chording keyboard to the pile of attempts. And I can personally obtain all the benefits that I wanted when I started down this road.

So I'm going to put a lot of effort into designing this keyboard for virtual reality. To start it off, I'm acquiring ~$500 in funding from my school (the program is called TechX, for the curious). In exchange, I'll develop a polished prototype by the end of January and present it to fellow students and the representatives of many technology companies. Last year I did the same for a different project and ended up in a booth next to Oculus Rift. Hopefully that happens again this year...

Reflecting the changed direction and renewed vigor, I've removed the old project logs. They are archived here: owen-t.me/keychange/paleotype.html