Project Logs

Collapse

• Program Context Driven Shortcut Overlays

  thpoll • 05/24/2024 at 16:53 • 0 comments

  For a long time, I have been talking about this topic, finally, it's here and I love it!

  The missing piece was the code that loads overlays depending on the active window. The mapping is stored in a yaml file that looks like this:

  Gimp:
    overlay: overlays/gimp_template.png
    app: gimp-2.*
  Inkscape:
    overlay: overlays/inkscape_template.mods.png
    app: inkscape.*
    title: .*Inkscape
  KiCad PcbNew:
    overlay: overlays/kicad_pcb_template.png
    app: kicad.*
    title: PCB Editor

  The fields `app` and `title` are regular expressions (Python). If these match, the `overlay` will be loaded (which can be a list of entries a la [a, b, c]). At least one of the two has to be present. The first entry that matches `app` and `title` will be displayed. The name/key of the block is just used for debugging.

  The overlay can be png, gif, jpeg, bmp or whatever Qt allows to load (currently it needs an alpha channel all the time, but this restriction should go away, however, I recommend png as it has lossless compression and you can always have an alpha channel).

  The white pixels will be read and turned on. Everything else is ignored. If the name contains ".mods." like "overlays/inkscape_template.mods.png" then the code will split the RGBA channels to the following scheme:

  R: shortcuts to display with CTRL
  G: shortcuts to Display with SHIFT
  B: shortcuts to display with ALT
  A: shortcuts to display when no modifier is pressed

  In case the name contains ".combo.mods." then the shortcuts will be used for the following modifier combinations:

  R: shortcuts to display with CTRL + SHIFT
  G: shortcuts to Display with CTRL + ALT
  B: shortcuts to display with ALT + SHIFT
  A: shortcuts to display with GUI key (not yet, hopefull y soon)

  For now, I decided to not support CTRL + ALT + SHIFT as it is rather rare, but you can try to convince me (pls. write a comment).

  Save the pixel color values even if the alpha channel makes it transparent and set the format to 8-bit RGBA. If you export a png in Gimp there is a dialog to do so (always set "Save color values from transparent pixels"):

  (I first used the Gimp command `decompose` to split up to RGBA channels and before saving the command `compose` can be used again to make a single image out of it - make sure that there are no other channels present, otherwise compose will fail). You can take a look at my gimp files in the "overlays" folder as a reference.

  What's now missing, are more overlay files with all the icons needed. I now have the following:

  • Chrome
  • KiCad
  • Gimp (partially, there are really a lot!)
  • Inkscape
  • JetBrains IDEs
  • VS Code

  Would be great to have an automatic way for the icon collection...

  One More Word On Platform Support

  While my current Python library `PyWinCtl` (to make program context shortcuts possible) supports active window reporting for Windows, MacOSX and XDesktop, it lacks an implementation for Wayland.

  Today, I came across this sample https://github.com/luisbocanegra/plasma-cursor-eyes that uses a KWin script to report the cursor position to Python, changing that to report the active window should work in the same way, so I'm looking forward to that!

  Enjoy your weekend,

  Thomas

• How The Protocol Works

  thpoll • 04/21/2024 at 16:46 • 0 comments

  To communicate with the PolyKybd, I can depend on QMK to give me HID messages, the so-called reports. All I have to do is to add the raw_hid_receive callback:

  void raw_hid_receive(uint8_t *data, uint8_t length)
  

  Even though the callback provides a length parameter, all reports have a fixed size of 32 bytes. For larger packages, you must slice your data on the host side and transfer it in junks.

  I started my protocol with a single identifier byte 'P' (because why not) followed by the actual command.

  For instance, P0 means: send your identification. P1 means: Send me your currently active language. The PolyKybd answers with "P0.PolyKybd Split72" or with "P1.EN". The dot after the command character means that the command has been understood (the first 2 bytes are always echoed). In contrast, the keyboard replies with "P1!" if there is a problem providing the requested data. Simple :)

  However, when supporting VIA (eg. to configure the keyboard layout via the web-browser), I have to stick to the VIA convention. Here, the first byte is already the command and there are 3 different kinds of custom commands (instead of a 'P'):

  id_custom_set_value = 0x07
  id_custom_get_value = 0x08
  id_custom_save = 0x09
  

  If the host sends any of these 3 reserved commands VIA will not do anything, but pass on the data to the via_custom_value_command_kb callback:

  void via_custom_value_command_kb(uint8_t *data, uint8_t length)
  

  So my solution is to accept either the 'P' or 0x09 (in case of VIA) and the rest can stay as it is (I’m not supporting the 2 other custom command bytes as they are meaningless for now).

  So what are the other commands next to P0 and P1?

  
  

  P2:

  Send a list of supported languages, which is a comma-separated list that might span over multiple reports:

  P2.EN,DE,FR,ES,...

  
  

  P3:

  Change the language - followed by a byte with the language to use (the index refers to the order as the languages were reported with P2)

  
  

  P4:

  Send a key overlay - followed by a byte for the keycode (so the key mapping) and another one for the modifier key (is this overlay active with Ctrl or Alt?) Since multiple reports ( 15 ) are needed for a single overlay (72x40px), there is one more byte indicating which of the 15 segments is currently sent.

  
  

  P5:

  Reset the overlay buffer (in fact just the usage buffer).

  
  

  P6:

  Followed by a byte to enable (anything != 0) or disable ( == 0) the overlays (whatever is in the current overlay buffer).

  That's it for now, but I will need more commands soon.

  
  

  Refactoring

  As I had to transfer the overlays also to the other side, I decided to refactor the communication code. It was already hard to maintain.

  It grew complex since I had issues with some of the synchronized data having toggled bits. So I added here and there mechanisms to double-check. It was never bullet-proof and once in a while, some of the displays didn't turn off (plus other small issues).

  At some point, I added a crc32 checksum to my custom messages and was surprised to see that there were failures.

  First I suspected the cable but it could be also related to the PCB layout. Or could it be something else?

  It took me ages to realize that also QMK split communication uses a checksum (crc8). But I'm not sure if that is done on all data packages. I also suspect that crc8 is not enough for longer messages. While I never saw wrong keys reported from the slave side, I could see the wrong layer information for a fracture of a second on the slave side.

  When finally refactoring, I got rid of my extra checks and just added a crc32 to every message sent. In case of an error, I just send it again. Layer information, modifiers etc. are now transferred with my own messages (and the QMK split transport for those is disabled).

  With that change, I couldn't see any issues like the ones I had before. Even after setting the transfer speed to 2 times faster, everything works fine.

  Now adding a piece of code for transferring the overlays to the other side was rather...

  Read more »

• Shortcut Overlays!

  thpoll • 04/02/2024 at 21:30 • 0 comments

  Alright, so this was a bit harder than expected and I was suffering quite a bit!

  Nevertheless, I'm glad to show you the first program shortcuts that have been sent from the computer to the keyboard:

  Some of you might have noticed right away: These are GIMP shortcuts and as I found out... They are quite useful! I looked them up on a cheat sheet first. Not anymore!! 😁

  Its satisfying to see the last of the 3 use cases (next to different languages and arbitrary layouts) I originally planned, come to life.

  Of course, there is still a lot of work to be done, but as usual step by step.

  I organize the shortcuts in images with space for 90 (10x9) keys with each having the full 72x40px resolution. By intention, I decided to pick the full resolution and not just some space for the overlay as this host-driven fully graphical approach feels more powerful than the text-based rendering from the PolyKybd itself. With that, I could do a lot of customizations (different fonts, text sizes, etc...) on the fly without pre-programming it into the firmware.

  Here is my current example:

  It's not complete, but it works for my current test scenario.

  There are 3 more images needed as the shortcuts usually differ if you press CTRL, ALT or SHIFT. Maybe I put just all together into an image using the RGB channels for the version when pressing the modifiers and the alpha channel for the shortcuts without modifiers.

  So why was it so difficult? Good question. I chose Phyton for the host program, so it can be easily changed by everyone and can potentially run on every platform without the need of compilers etc. While I did some work with Python in the past, I never had to deal with image manipulation before.

  This is, where my problem started. Without getting into too much detail: It was not easy to find images libraries that work on Windows, Linux and Mac, that have no namespace collision issue a la PIL and Pillow and run together with Qt (openCV doesn't). I settled on the obvious choice of NumPy and used imageio for loading the image data instead of PIL. Maybe all the Python wizards are laughing now and could have done it during lunch break, but believe me when I tell you it took me multiple days (sure I'm not working on this full-time). Besides that setup issue, it also took me some time to prepare the data (slice the image to the individual overlays, encode them as monochrome bitmaps, transfer only entries with actual content...) so it can be understood by the keyboard and its HID protocol.

  In case you want to take a look at it: https://github.com/thpoll83/PolyKybdHost The most recent firmware will gladly accept the overlay, but not transfer it to the other side, that is still something I have to do.

  Be warned that I just tested it on Linux and it is not intended for a real user.

  And Here Is Some Other News!

  I will talk about the PolyKybd at Hackaday Europe 2024 on the 13th of April, 3pm!! The lineup is not yet announced, but I will add a link as soon as possible. Let me know if you will be there as well!

  Best, Thomas

• Prototypes are out for Testing!

  thpoll • 03/15/2024 at 19:22 • 0 comments

  Right, so there are 4 prototypes in the wild! The first one will stay at Crowd Supply for pictures etc, so actually not so much for testing, but more as a reference and display. The second one just arrived today at my most loyal supporter Chad. Hope you enjoy putting your kit together! Of course, I am very curious to get some feedback from you :) The third PolyKybd is currently in Germany. I handed it over to 0x17, who organized CCH Con this year in Reutlingen. Hopefully, he can pass it on to some other testers soon. All these boards are hardware revision 2.2.

  The last one is a ...ahm... a "Frankenboard" with one side having revision 2.2 and the other side is a revision 3.1 board. It stays with my good old friend cbirklbauer as my most and highly trusted software engineer of choice.

  I hope your comments and observations will improve the software quality which is still a bit rough, but we will get there :)

  So what is different with revision 3.1? Not that much! I exchanged the RGB LEDs as the WS2812B-Mini-V3 a lot of times had manufacturing issues. With the new XL-3030RGBC-WS2812B, I haven't had any issues and they appear to be a bit brighter. Another component, I exchanged, is the socket for the 14pin ribbon cable of the displays as the original part has a production delay of about half a year.

  The only real functional change is a fix for the USB-Sense. With that, the keyboard can tell, which side is connected to the host (and then handle the logic and communication). In revision 2.2 the software does that by detecting a USB connection. That works in most cases, but maybe not for some older BIOSes or in case you power up the keyboard via an active USB-Hub without a PC connected to it. This should be fixed in 3.1 as I can assume that the side with the power, will be the host side.

  Just a few days ago, I assembled the first two PolyKybds with both sides on revision 3.1:

  A low-profile version with Tecsee Medium switches.

  
   

  And here is an experiment where only R1 has low-profile switches. Let's see how comfy that is...

  Other than that, I had to do some number-crunching to make sure, that the Crowd Supply campaign will be financially sound. There is nothing I can show you, but these tasks also have to be addressed. The only productive side-task I had, was the construction of the legs:

  Looks like a fit:

  If these legs are not enough, I encourage you to come up with a custom tenting solution.

  For that, there are a couple of holes on the back and some pre-perforated holes on the side, which you could drill or break out. Either glue in some threaded inserts or just a 3D-printed stand/leg of your choice!

  Best, Thomas

• OLED burn in?

  thpoll • 01/28/2024 at 16:03 • 0 comments

  Of course, we all heard about that, there are plenty of articles and videos out there.

  Maybe the more important question is: Will this be a problem for the PolyKybd?

  To find out let's take a look at my current Prototype, which I now use for 3/4 of a year - I even bring it to the office, so it is used all day long at least 5 days a week.

  Before looking at the first picture let me mention that you will see some lines flickering (some lines appear lighter on the pics) here and there because of the refresh rate of the display not being synced with the camera sensor.

  Here, we see the left side when not pressing any key:

  Let's focus on the 'd':

  And now let's compare to the upper case 'D':

  Notice any difference in the brightness apart from the flickering lines? No? How about here:

  Same key, also here:

  As you can see, you won't recognize little brightness differences on specific pixels of the glyph.

  But where is the burn in effect now? Alright, of course, I can forcefully make it visible. For instance, if I invert the whole screen, like that and adjust the brightness and contrast of my camera:

  Now you can see the lowercase 'd' which is displayed most of the time (+ some flickering at the bottom).

  So there it is! Yes...

  Will it affect your daily work? See the next pictures taken from 15cm (~6 inches) away from the keys (so multiple times closer than your eyes - at least I hope that's the case):

  So at this point, I would say: Maybe not.

  This is great news. Additionally, let me post similar pictures in a while again to get some more empirical data!

  Nevertheless, I'm doing my best to protect the screens:

  1) After a minute of inactivity I switch to the idle mode, or as other people call it: "The screen saver mode": In that mode, the displays turn on and off randomly with very low contrast and usually just a few of them at a time. It should give you enough clues to find the right key while avoiding permanently turning on all displays.

  2) After another 5 min (or whatever you set) all displays turn off and wait for the next key stroke. You can configure whether that first key stroke should be accepted and forwarded to the PC.

  I tried one more measure: To randomly offset the displayed characters by 1 or 2 pixels in x and y. However, that didn't look satisfactory as you can notice little offsets very well.

  What do you think? What's your impression? Have you ever had to deal with OLED burn in?

• Testing another 3D printing farm and preparing a first kit

  thpoll • 01/28/2024 at 16:01 • 0 comments

  From the beginning, I have been using 3D prints for the PolyKybd cases and other small parts and was always satisfied with the quality of these resin prints. As there are already plenty of options, I wanted to see if there is a 3D printing farm in Europe comparable in price and quality to my go-to-shop JLCPCB:

  In the picture above, you can see that all the small parts from JLCPCB have a good surface finish, there are hardly any marks from support structures, and the dimensional stability is impressive. Especially the last point is important so that the MX stem firmly fits.

  So I went off to https://craftcloud3d.com (where you can get offers from various 3D printing farms all over the world) and indeed found a company that would offer similar prices for 3D prints with higher order quantities in Europe. Even if the prices are slightly higher, there would be no import toll within Europe in contrast to anything imported from China. With that in mind, I ordered a smaller batch (which is unfortunately quite expensive compared to high-quantity orders) to see how that works out...

  While the support was quite responsive with questions before starting the 3D prints, it took longer to ship the prints from Sweden to Austria than from China to Austria :'(

  However, I'm patient, and waiting a week longer is not a big deal. When the parcel finally arrived, I opened it and was quite disappointed: Straight corners were bent and twisted, the surface showed a lot of spots from the support structure, there were seems and when measuring with the calipers it was clear that the prints were unusable: 

  Even more visible on the cases:

  Comparing to:

  Of course, the cases are different iterations, but I believe you can still spot the quality difference.

  I reached out to the print farm and they told me that they had an issue with their resin that was discovered later, so I gave it one more try and they sent me a batch of replacement parts free of charge:

  While the dimensional stability improved, there were very ugly break-outs where the support structure was attached:

  Also, the surface finish is still far away from JLC, which is very visible in direct comparison:

  It might be worth trying one or two other farms, but that first experience was not the best.

  In the end, I got my 3D prints from China once again and put my first test kit for crowd-supply together:

  At some point, I should think about the packaging. For now, I just put everything together with bags, trays, and wrapping I had at hand.

  I hope you get a good rest during this holiday season and wish you all the best in 2024!

• PolyKybd Split72 Kit will launch on CrowdSupply!

  thpoll • 10/06/2023 at 16:58 • 0 comments

  Yes, right! Look at that screenshot! The PolyKybd is at the top right!

  As of today, there is a pre-launch page live at CrowdSupply, where you can register to get all updates on the campaign for the kit: https://www.crowdsupply.com/polykybd/polykybd

  Of course, there is still much to prepare before the actual launch of the campaign and I would like to come up with some (long time ago requested) typing videos. Nevertheless, this is the first step in making this whole project a reality - not only for me but for you folks out there as well.

  
  

  Why Crowd Supply?

  There are plenty of crowdfunding platforms out there and every one of these has a different target audience. Crowd Supply is maybe the most hardware-centric platform, where backers have a good technical understanding. And this is exactly what I'm looking for: Folks who are not scared of flashing the firmware, folks who can use a screwdriver! I want everyone who buys a kit to be happy with it, even if you have to tinker a bit to make it fit to your needs - be it the software or the hardware by adding a rotary encoder. After all, we are individuals, right?

• Instructions are Ready!

  thpoll • 09/09/2023 at 20:13 • 0 comments

  Yes, finally I got the build guide ready on my HW github repo: https://github.com/thpoll83/PolyKybd

  I wanted to wait for my first batch of pre-production displays to be showcased in the instructions (along with the regular soldered ones, you have the choice) so there was some delay. The first batch:

  Around 500 displays, all with long enough FPCs to directly connect to the socket! No more FPC-toFPC soldering needed :)

  With that, the build guide is finally out! It contains a lot of pictures and was more work than expected. If you are impatient and you do have a hot-air rework solder station, you can go ahead and start putting a PolyKybd Split72 together :)

  If you don't feel confident enough for the FPC soldering part or you don't like to buy all the different parts across various sources from the internet... Stay tuned! As you may know from previous post I'm trying to put a kit for PolyKybd Split72 kit together, just don't expect it next month or this year for Christmas.

  And I can also tell you that the new displays are much smother in operation compare to the old ones, where I extended the flex cable!

  What else happened?

  Besides a little summer vacation (yes I need a break from time to time too) a VERY similar project came up: https://www.kickstarter.com/projects/daptkey/daptkey-redefine-your-life-with-a-customizable-keyboard/description

  They "had" the idea withe the OLED displays in October 2022🤔

  Not sure what to think about it. They use James Brown's mechanical solution of a separate slot in front of the key switch and the flex cable comes out at the bottom edge of the key cap. I do wonder if there is any mechanism implemented to reduce the OLED burn-in? From the video, I would rather say no. Granted, the look of the case and caps have a nice finish and these guys moved quite fast from development to production.

  What are your thoughts? Let me know in the comments below!

• Customized Displays Arrived!

  thpoll • 07/14/2023 at 10:58 • 3 comments

  Here they are! It maybe doesn't look like much, but this is really a big step forward: The first 2 prototypes of the displays with the extend length FPC straight from the factory.

  Maybe not such big news for all Ko‑fi supports, as I already announced the work on this in a private post some time earlier. Still, I'm happy, that I was able to get these 2 prototypes and soon a few hundred pre-production samples without ordering 10.000 pieces right away!

  So, here you can see the first one fitted into my prototype with all the other displays:

  By intention these are just a bit shorter than the manually extended ones to reduce the needed space below the PCB, since there is even more space requirement when pressed. And I'd like to avoid the cable touching the case as it adds extra resistance when pressing:

  There is still some wear from the rough edges of the FR-4 slot and I've been toying around with plastic inserts to give it some more protection.

  However, that didn't lead to a good solution yet and it might the easiest to just a apply a bit of thick paint/coating around that edge.

  From the outside the new displays look almost identical:

  (first row in the middle)

  From the first picture you might have noticed that there is no hatched copper fill on the backside of the FPC. With the absence of that, the flex cable got much smoother and the typing experience in my opinion improved a lot now :)

  
  

  Keycap Orientation

  Apart from these great news, I have been experimenting with the orientation of the keycaps:

  I find it a bit difficult to always find the right row, if all keys are uniformly flat. There is a little notch for the index fingers, but still I do hit the wrong key from time to time. This could be a matter of taste,  but I also hope to improve the ergonomics with this little tweak. In the end everybody can choose the right profile according to personal preference. Here a photo from the second prototype I'm putting together:

  The next possibility that comes to mind would be a slight step between each row like most commercial keyboards have:

  In that case you might need some tenting. Or you just come up with you own version. The stems are made in OpenSCAD:

  https://github.com/thpoll83/PolyKybd/blob/master/keycaps/keycap_stem.scad

  It should be rather easy to customize. The only limiting factor is the length of the flex cable. 

  Repository Cleanup In Progress

  Yes, the long promised cleanup of the repository is in progress an I'm writing a short build guide. More on that next time..!

  Thanks, Thomas

• Finally, After 2,5 Years!

  thpoll • 05/24/2023 at 05:06 • 0 comments

  It is finally working! Both sides can be used without modifications! 😁

  But that is only for the hardware of course, software is never done as we all know!

  
  

  Current Software Work

  The code is already much cleaner and my current schedule is about testing through the optional devices like rotary encoder, pimoroni trackball and cirque trackpad:

  - The rotary encoder works.

  - The pimoroni trackball needs a QMK update as it has some issues in my version (the cursor becomes very shaky). To be done soon...

  - The 23mm cirque trackpad works after resoling some configuration issues. It is a bit small, so I might try the next size as well.

  
  

  Open Software Issues

  - The status display on the right side (slave) turns on unexpectedly after the turn-off timeout (just resolved minutes ago, so this is also done)

  - An issue I had in the past: RGB LED matrix flickers on the master side. Maybe because I changed the LAYOUT macro to reflect the key orientation..? Need to check if I have to do the same for the led_config.

  - Easy way to configure the layout. Right now I still have to flash the firmware every time, but at some point I would like to switch to something like Vial so that can be done without flashing anything.

  - Some more Icons for all the special function keys.

  So you see, there is still plenty of work left.

  Further Software Improvements Ahead

  - This is out of scope for now but definitely the goal: To have a program on the host side to switch the OS input language and also send program context information to the keyboard so that customized symbols can be displayed.

  - To avoid burning out the OLED displays, I plan to shift the displayed content slightly once a while. This is nothing fully developed yet, more an idea.

  
  

  Final Touches

  The PCB is in a production ready state, however I would like to apply a cosmetic change to the silkscreen (some pin markings are on the wrong side - luckily only on the silk screen).

  And this is more about the general mechanical design:

  If possible, I would like to have some plastic inserts for the flex cable slots:

  With that, the edges would be soother and make the flex cables maybe life longer. I had no such issue so far and the edges are not rough, but if it is possible, I'd like to try. It sounds like an improvement to me. I was also thinking about plated slots, but these have size limitations which makes it impossible (at least at JLCPCB).

  
  

  Layout

  Of course everyone can come up with an own layout, but let me quickly show off my current choice (and might change soon again):

  I started with a symmetric layout similar to Ergodox. With the number keys 1~5 on the left side and 6~0 on the right and in a similar fashion distributed the letters.

  However, I'm really used to the traditional stagger and there, the left and right hand finger assignment is distributed rather un-symmetrical. Maybe this changes at some point, but for now this layout is much easier to type!

  Your perfect layout might look different... let me know in the comments below~

View all 32 project logs