FREKVENS FJÄRRKONTROLL

Details

Rendering 16x16 pixel graphics at 60 FPS via JavaScript on a hacked IKEA FREKVENS LED light.

I embarked on this project just about a week before Toronto went into shutdown due to the COVID-19 pandemic. I had already purchased the FREKVENS cube, with plans of doing something with it, while not being quite sure what. The entire project ended up being realized with just materials that were lying around. I mustered the courage to get cracking (and also kinda waited to share the fun, and also rely on his expertise) one weekend when my good friend Nick Matantsev came over. We pried open the box together, while somewhat exercising social distancing (Again, this was a week before things around there got really serious).

And it organically evolved into this Rube-Goldberg-like contraption, taking a long path to just rendering an animation on an LED screen:

The author writes a time-based animation scene in JavaScript on a minimalistic rendering runtime using a web interface
Upon submitting, the JavaScript is broadcast to rendering clients via WebSockets
Inside the FREKVENS cube is a WiFi-connected Linux computer (Onion Omega 2+) that runs a Node.js daemon that picks up the JavaScript scene and starts sending rendered frames to a Node.js extension written in C++ (aka the driver)
The driver uses GPIO to send pixel data to the LED controller
All the clients send time synchronization requests to the server to be aware of the time delta to the one and only physical rendering client (the cube)
Regardless of which screen (or the physical cube) you're looking at, all the rendering is in unison

All the code is open-source and the GitHub repository links are in the project details.

The original cube before disassembly:

This is the LED controller (white) and the graphics controller (green) inside:

This is after replacing the graphics controller with an Arduino Uno prototype, timing code and electronics courtesy of Nick. Info on pinout + data transfer provided by Brent Marshall was also pivotal:

I continued the project on my own after that initial night of hacking.

Switching to an Onion Omega 2+, rendering some test patterns:

Some case modding to pass in a USB-C cable to power the Omega:

All the guts stuffed into the cube, with some really gnarly soldering job:

Cube closed, ugliness hidden:

Custom hole with the USB cable sticking out. QC passed, warranty void:

Rendering some Sierpiński triangles as a test animation, with a piece of red paper taped to the face as a filter to avoid getting blinded by the extremely bright LEDs:

I also repurposed the buttons on the back:

Red button: blanks the rendering on short press, powers off the Omega on long press
Yellow button: cycles between different scenes on short press, reboots the Omega on long press

There are also status overlays rendered on top of the scenes:

A blinking pixel on the top left indicates reconnection attempts to the WebSocket server
Button presses are rendered as large circles (corresponding to the exact positions at the back)
A pixel on the bottom left indicates a CPU choke or frames being dropped

Because writing JavaScript on an SSH connection to the Omega and then to restart the Node.js script was very cumbersome, I created this toolset to make pattern creation faster and more enjoyable:

A WebSocket server running on Glitch
A WebSocket client that emulates the cube, including light diffusion, and mirroring the rendering to the favicon for no reason, rendering on an HTML canvas
A text mode emulator that kicks in when the Omega driver is run locally, again emulating the light diffusion in full ASCII art glory
All clients time-synced to the Omega (continuously compensating for the latency of WebSockets over the Internet), rendering the same frame at the same time, with astounding accuracy

A happy tweak was to connect the USB power back to the cube's own power supply by using an old Apple iPad charger I found...

Components

1 × IKEA FREKVENS LED multi-use light https://www.ikea.com/ca/en/p/frekvens-led-multi-use-light-black-50420353/

1 × Onion Omega 2+ https://onion.io/omega2/

1 × Micro USB cable To power the Onion Omega

1 × Apple 12W USB Power Adapter Or any USB adapter with an IEC 7 socket

Project Logs

Collapse

Another demoscene tribute
Ates Goral • 05/23/2020 at 04:42 • 0 comments

Another timing test or just an excuse for blinkenlight coolness.
This is Masagin - NVISION 08 Invitation" by Farbrausch.
I screwed up and recorded this only in 720p, losing a lot of detail.
Large videos and streaming!
Ates Goral • 05/22/2020 at 22:02 • 0 comments

Made some good progress over the last few days.

First, I switched to on-demand loading/decoding of frames so that I can store and playback very long videos on the Omega. Here's the first successful test of playing back a 78 minute video:

Since first exporting video frames as CSV and then running them through my encoder to turn them info ".ff" files was proving to be very cumbersome, I wanted to experiment with streaming.

Streaming frame data from TouchDesigner over UDP and adding a UDP listener scene was a breeze. Here are the fist successful runs. First, locally, on the text-mode emulator:

Next, streaming remotely to the Omega:

And finally, once I got streaming working, it was trivial to hook things up to my webcam in TouchDesigner:
Accurate frame timing at last
Ates Goral • 05/20/2020 at 17:02 • 0 comments

My overnight experiment of running this binary clock for 8 hours and checking for frame drops / clock drift confirmed that I now have proper frame timing!
Even better frame timing
Ates Goral • 05/20/2020 at 04:53 • 0 comments

Significantly improved frame timing with a simple tweak to prevent timing errors for individual frames to accumulate over time.

Created a binary clock (to avoid the hassle of rendering actual digits). It's all 6-bit values from top to bottom: hours, minutes, seconds. The rapidly moving dot at the bottom is a frame counter: 12 x 5 = 60 frames per second. The clock doesn't read the current time of the OS constantly. Instead, it calculates what the time should be based on the rendered frames. If there's any timing issues with rendering or if frames are dropped, it should cause the clock to lag behind.

Showing reference time on my laptop with a big clock I wrote as a dweet and there's a 1-second difference despite both the Omega and my laptop being synched with NTP. I'll keep this running until the morning and see how much of a deviance there'll be.
Improved render timing
Ates Goral • 05/16/2020 at 07:09 • 0 comments

Improved render timing by moving it from relying on a JavaScript setInterval() to relying on a render event fired by the render thread. Looks much better. (I'm also getting a bit better at video production!)
Crunching frames
Ates Goral • 05/15/2020 at 22:48 • 0 comments

So far, I've been mostly focused on real-time rendering using JavaScript. For more complex scenes, or to quickly prototype visuals, I was planning to find a way to play videos.
Instead of some streaming setup, I decided to store the videos directly on the Omega, especially since I have lots of room in the form of a 64 GB SD card.
I created a TouchDesigner setup to extract 16x16 monochrome frames from videos -- saves each frame as a CSV file. I then wrote a Node.js script to take the CSV frames and encode them into a proprietary video format (I call "FRFR", stands for "Frekvens Frames"). Code and documentation is on GitHub.
I've been dying to see what one of my favourite demoscene demos, Intrinsic Gravity by Still, would look like on this limited medium. My initial plan was to try to recreate the scenes in JavaScript. But, directly stealing frames from a video of the demo, I was able to quickly test out what it would look like without writing a single line of demo code.
There's are some timing issues I have to look into.