believe it or not, i have been working on it for the past 6 months, even though there was ne log update. I was mostly doing the very boring stuff, making simple things like controll mapping and stuff. I was programming everything on the computer, and a few weeks ago it was time to try and make it work on the pi. I had to create a HAL for the pi, and after searching and fixing some very nasty bugs that only occurred on the hardware, i finally got everything up and running. It was a bit of an educated gamble if it would run good on the hardware because i had no idea if there were any serious performance issues on the hardware. I was developing on an arm that much much better, but was hapoy to find out that everything worked really well. Right now im finishing the file io access api of my little "not-an-operating-system", after that, i should start to implement a midi api, and then i can finally focus on actually making something that you can use, the samer itself. I have already made a waveform control with zooming and all. I'm looking forward to giving more info, but at the moment i dont feel i have a lot of usefull stuff to tell.
I've been busy the last few months, but have not been talking much. Radio silence does not equal "no progress"! My modular casing system for prototyping is ready, and am very happy with the results. I will be documenting the process when i feel like it's ready for the public. Talk soon!
Note: this is not the final control layout. One of the reasons why i developped my own 3d printable modular case system, is so that i can easily change and rearrange it while developing.
It's been a long time since the last log. A lot has happened for me in this time, and there was no room for openSampler in my schedule. I'm typing this on the train to work, so forgive me if this log contains more typing errors than usual. It makes me think about just how bad mobile phone keyboards are. I keep making the same mistakes because of muscle memory, shurely there must be some ai algortihm to assist me in this day and age...
Anyway, gonna keep this short. I only have about 20 minutes to finish this message. Long story short, i changed from fully self employment in 2020 to 80% employment doing my all time dream job at Cinematek, wich is the Royal Belgian Film Archive. I was lucky enough to be hired as an expert in film restoration and digitization. Fantastic working with film, but also very satisfying cleaning them up digitally. I'm sure many would find this is one of the most boring jobs they can imagine, cleaning dust all day, but to me it's heaven. I'm also buying a house right now, about the worst possible timing. The housing market in Belgium is being flooded by prospects looking for one, and stories are floating around that people are even buying houses just looking at the pictures without visiting them first. That's crazy. My motivation is different, but I managed to find something I really like. So this is taking up al my time for the moment. Well, not all of my time, but I seem to be having difficulty focussing on multiple big projects. I'm constantly thinking about the house project, and sometimes openSampler, but don't feel like working on openSampler unless i know that i can dedicate some time on it, and not just 10 minutes here, 15 minutes there.
So, what about it? Last time i think i talked about getting it to work on the twatch, well it all worked out. It's working well. Yesterday i finally spent some time setting up my dev environment om my new macbook m1. Yeah i bought one, but only after witnessing it at Cinematek where i tested editing 10bit h265 6k anamorphic footage from my gh5, and it worked like a charm. I was so amazed, that I bought a macbook air because my old one could not even play back the videos in quicktime or vlc. Anyway, the m1 comes with it's problems. I installed the parallels desktop technical preview and installed the arm64 version of ubuntu. All works great. Then i installed some dev libraries, nothing special there. Then, i got the the esp-idf. It's not officially supported on arm (yet, they seem to be working on it), so i had to do some tinkering to get it to work. Basically it came down to: compile crosstool-NG myself, register the path, modify the esp-idf install.sh script to bypass the xtensa installation (because it would block you from continuing because there are no precompiled binaries for arm), and then i also had to modify idf-tool.py to skip the checks for the xtensa compilers for the s2 and s3 versions. I dont need them. After that idf.py worked great and i got to compile. Then i tried to flash the watch, but i ran into some permission problems and had to chown the /dev/ttyUSB0 device before it would work. After that eureka! By the way, make sure your vm has more than 2gb of ram, because else the xentsa gcc will not compile and fail at the step installing gcc final binary or something like that. It will say something vuague like the process was killed. I changed my vm to 4gb, and all is fine.
I also did some home cleaning in the project, made sure everything is nicely seperated. Now i need to implement mouse and keyboard events, and then the ui system should be usuable. Can't wait until this relatively boring task is done, but i'm almost there.
I'm also gonna try and get the gcc compilers for circle working on the ubuntu arm vm, but i dont think it will be a problem, it looks like it already has good arm support.
I just couldn't help myself. Remember the lengthy talk we had about perfection and OCD in one of the previous logs? Well, the demon strikes again. I was busy with work, that made me work on a modular casing system that is about 90% completed. I have to wait for some parts to come in from China, so that's put aside for now. Then I got sidetracked because I found this Hyperpixel 4 display, and that used up all pins on a Pi, so I decided to rewrite my graphics layer to allow for remote commands over i2c. And while at it, I decided I would do it in plain C, because then I can reuse my graphics layer on the esp32. I know you can use C++ on the esp. I just prefer not to.
I just bought this briliant watch, the Lilygo TWatch 2020. The graphics layer is also about 90% finished, and ready to be used. Then I thought, gee. I have all these ESP32's lying around and have got nothing to do with them. Wouldn't it be awesome if I could make some kind of a micro sampler reusing my existing (simplistic) audio engine? My Pi with touchscreen is a bit bulky to develop on and take with me for developing on the road, so I decided to go for it. First I made it work on the internal DAC, then using and external DAC. Then just for fun, I tried to get it to run on the watch. After some fiddling I realised that all examples for this watch are written with the Arduino framework. I'm using esp-idf, so I had to figure some things out. I discovered the reason it didn't work on the watch was because the DAC was not powered on when the esp32 boots. Makes sense, it's a watch, power management is very important there. You need to talk to the power management unit in order to power it on. I traced the code required to get it to power from several examples online. Once that was done, it was working on the watch as well. You would think the watch would use the internal dac to connect a speaker, but they decided to go with an all-in-one package i2s codec + amplifier.
Oh boy, that feeling of accomplishment... You see, I fell down this rabbit hole of side-quests. But I gained a lot of experience in doing so ;-). I also learned that I could get about 20 mono voices with pitching and gaining to work on a single core of the ESP32. Impressive, i thought it was gonna be a lot less.
Anyway, the plan is now to work more on the engine, so i hope to see some progress in there. I'm also still learning how to properly structure and organise a project targeting multiple hardware platforms, so unfortunately there will be a lot of refactoring along the way. But a tidy house, is a nice house :-)
Short warning for all those who are interested in using the HyperPixel 4.0 Square display: it uses ALL the GPIO pins. In my experience not even the UART is accessible anymore. That's a no-go for openSampler, but I still want to use this display because the dimensions are absolutely perfect. I'm thinking about using the pi zero to drive the display, and let the pi 4 (or the cm4 when it comes out) send display commands over i2c to the pi zero. This way I can also use the display with any MCU that can talk i2c, such as the esp32. I'm not the first one to do such a thing, as far as I know, NerdSeq also uses the pi zero to drive a display.
For those who also want the use this display in bare metal; you need to add some lines to the config.txt, and add the hyperpixel4.dtbo file to the overlays directory on the boot volume. This file needs to be built by the install script. They used to have a compiled version in their github repo, but it's gone. I don't know why though, I 'compiled' it on a Pi 4, and tested it by using the same sd card in a pi zero and it worked without modification. If you need it, you can find my compiled version here: 'https://gitlab.com/nickverlinden/hyperpixel4-baremetal/-/blob/master/doc/pios-files/pi4b/boot/overlays/hyperpixel4.dtbo'. You also need to do some display init code in your project. The code can be found in this file: 'https://github.com/pimoroni/hyperpixel4/blob/square/src/hyperpixel4-init.c'. You need to do some rewriting so that it uses circle's GPIO functions. The code basically bit bangs some display init commands over one of the GPIO pins. I think it's a little weird because it looks like it's some sort of SPI, so I don't know why they just didn't use the SPI bus. If someone knows, shout it out in the comments please.
This is the output of the install script from hyperpixel, all the lines added to config.txt are of interest:
Config: Added dtoverlay=hyperpixel4 to /boot/config.txt
Config: Added overscan_left=0 to /boot/config.txt
Config: Added overscan_right=0 to /boot/config.txt
Config: Added overscan_top=0 to /boot/config.txt
Config: Added overscan_bottom=0 to /boot/config.txt
Config: Added framebuffer_width=720 to /boot/config.txt
Config: Uncommented framebuffer_height=720 in /boot/config.txt
Config: Added enable_dpi_lcd=1 to /boot/config.txt
Config: Added display_default_lcd=1 to /boot/config.txt
Config: Added dpi_group=2 to /boot/config.txt
Config: Added dpi_mode=87 to /boot/config.txt
Config: Added dpi_output_format=0x7f226 to /boot/config.txt
Config: Added hdmi_timings=720 0 15 15 15 720 0 10 10 10 0 0 0 60 0 35113500 6 to /boot/config.txt
Just a quick update here. I've been very busy with work, which should be a good thing in these times. Unfortunately, that also means less time to play around. But... openSampler is always on my mind, and I have lots of time while commuting to think about the execution. It really helps not to take things too fast, because while staring out the train window, often an idea will pop into my mind, that will solve some problem I was having.
I've been working on a modular 3d printable case system in the evenings. It's nice, it takes some time to print parts, trail and error. The casing system will be used for openSampler. I have this idea that the controls for the sampler are completely customisable. If you want to have a rotary control, you can. If you want to have dedicated hardware buttons for something specific, you can. Heck, if you just want a touchscreen and no other controls, fine!
I'm thinking about making all control boards i2c modules. That way you can add controls as you please, or remove them if they're not working out for you, or even remap their function to something else. I'm pretty excited by this idea. In my head it's all easy-peasy; but I suspect the reality of it will be a little more complex.
Anyway, i'm focussing on perfecting the 3d printable case system now, next I'm going to try another touch display. I've bought this square HyperPixel touch screen, which seems perfect for me. I found the raspberry pi 7inch touch screen a little bit too bulky. I think the display will work out of the box, but the touch screen might not. It's i2c based, and think Circle's built-in touch driver is for the memory based touch screen only. i2c is relatively easy, so it should not be a huge problem to implement this. Let's see how that works out.
If you read the previous logs, you might have gotten the message that I love 1-bit guis. I loved Atari TOS and the early version of MacOS. To create 1 bit graphics, you can draw lines or pixels using code like this for instance (this is not real code!):
Draw::Line(0,0,320,240);
To draw fonts, there are several different solutions out there. Having followed David Welch's bare metal programming tutorials, I was using the psf font format, which is very simple. It contains a 32byte header with a few important parameters such as the width and height of the characters. You can create these fonts by hand if you want. Just create a C header filer with a byte array, and start entering bytes manually according to the spec.
Even though the format is quite simple, it's a very tedious job. I looked around, and there seemed to be no simple (free) app to create 1 bit pixel graphics. I also have a Samsung Note 9, and it seemed like a romantic idea to be able to use the included pen for drawing. I have to travel to work a lot this month, so I had some train time to waste. That's when I created bixl. It's a very simple pixel drawing program.
I had some experimentation code lying around for creating a pwa, so this web app is also a pwa that you can install directly from the browser onto your computer/phone/...
You can start from scratch, or load an existing psf font. Only psf version 2 is supported at this time, but I'm not planning to add support for more file formats. If you want to add it yourself, dig in, it's open source! :-)
Fighting the lifelong enemy called perfection
Now is a great time to talk to you about the worst enemy I have to face all the time: perfection. If you don't want a lecture about that, just skip the rest of the log as there will be no new technical information.
When working on stuff like this, it's easy to lose yourself in details. I constantly think like: 'oh it would be nice to add x feature', or: 'people will probably need feature y'. This process keeps on going and if you give in to it, you will never deliver or it will take you a very long time to do so. And even then, it will never feel complete. there's always something more you can add.
A few years ago, I was dreaming of a distributed system where you could just include packages directly from github/lab/... in a web app. Think of it like npm, but without the need to locally download and keep these libraries. Your browser would just fetch them when needed. So I started experimenting and writing code. At one point I had a working system. It was fast, and it worked. Then the ocd kicked in and said: you need to rewrite this now, because people are going to see this mess you created. So I started to rewrite with the knowledge I gained by creating it. Javascript promises where the new thing, and everyone was using it, so I had to use them in my rewrite as well. Then, that version was almost finished, and I decided I wanted to add a feature because it seemed nice to have, or it just made sense (though not required!). But then I patched something and the code seemed messy again (though it really wasn't, but I thought it was), so I decided I would rewrite it from the ground up with this new knowledge. By the end, you could write plugins for loading from your own custom repository and all, quite fancy. It just took 3 years of train travel time, and a decent amount of spare time from my life to create it. A few months later, this project was almost obsolete by the javascript import api. I never started using it. So when I look back at it now, it just seemed like one big waste of time. Image you spent your life building your own car. But by the time you finish, cars are no longer being used and were replaced by drones. You created a nice car. For yourself. Great. Now you know how to build a car.
So it was then that I realised that I have to fight the ocd and perfectionism when it strikes. With bixl, I had the original idea to also integrate a reverse engineering tool for inspecting binary files for 1-bit graphics. There is a tool that I created a while ago, and recently cleaned up a bit for public use. it's available here: https://synthology.gitlab.io/ceebit/. My plan was to add that to bixl as well. While I still very much like the idea, I have to restrain myself Both products exist, and are working. There would be no other benefit except to have both tools in 1, and to satisfy my inner demon. Here is a screenshot of a wip version that had an extra top bar with a drawing tab, and an inspect tab.
If you look at the source code of bixl, you will see that every line of code is inside the index.html file. Ask any serious web developer and this would be frowned upon. The way I work is usually like this: I have an idea, I start messing around and suddenly I have something that works. Then I'm supposed to rewrite it the way it should be done, or to say it with proper terms: refactor. Nicely separate the ui code from the logic and stuff like that. instead, this time I decided to skip that step and leave it like this. In my mind this is a complete mess. But you know what? it works. It does what I want, it's reasonably featured, it works on desktop and smartphones, it looks nice and it can load and save psf files. Why on earth would I want to start refactoring it just to satisfy parts of my brain that like things clean. I'm giving this away for free. Not that this is a ground breaking piece of software, but I spent time on it. Hours of my life that I could have spent otherwise, and I'm giving them away to you for free. Are you really going to tell me I should have cleaned it up before 'donating' it to the community?
If you ever have the chance and access to look at some commercial closed-source code, you might be surprised at what you would find. When having a beer with a good friend that works for a company that delivers software for the military, this subject came up. He told me something that stuck around. His philosophy was that If it works and it's stable, it works. The time you usually spend to clean things up afterwards and make it 'just perfect' is not worthed. Because even if you reach perfection, something is going to mess that up anyway. A new feature, some system changes, ... Try to spent as little time as possible rewriting or refactoring.
Even now, when working on openSampler, it frequently comes to mind: should I just start over and do everything in C? Because if I would use C, then I can reuse a lot of code for other projects that only have C compilers. That would mean I have to rewrite or even reimplement certain features of Circle, and that would be an immense time hog. No. I need to fight the idea. Just stick to what you have created already. If the problem comes up that I want to reuse some code on a microcontroller, then I'll just have to cross that bridge when I come to it. Why start rewriting stuff when you don't even know you're going to use them. And why would I even think I could do a better job than what already exists in the Circle library? I probably don't, and most likely make a mess out of something beautiful!
Failure is a part of life, and we just have to deal with it and move on. After several hours of trying to compile the aarch64 gcc cross compiler on macOS myself, I have given it up for now. I will continue development and testing in 32bit. I'll just compile and test on linux for aarch64 when a major version has been released.
I got as far as getting a binary for the g++ compiler, but using it with circle results in:
CPP actled.o
'armv8-a+crc' is not a recognized processor for this target (ignoring processor)
'armv8-a+crc' is not a recognized processor for this target (ignoring processor)
/var/folders/q1/0jsxyqlx6jl_592pvc_z3lx00000gn/T//ccLAjiBh.s:1:2: error: unknown directive
.arch armv8-a+crc
^
/var/folders/q1/0jsxyqlx6jl_592pvc_z3lx00000gn/T//ccLAjiBh.s:10:2: error: unknown directive
.type _ZN7CActLEDC2Eb, %function
^
/var/folders/q1/0jsxyqlx6jl_592pvc_z3lx00000gn/T//ccLAjiBh.s:17:16: error: brackets expression not supported on this target
stp x29, x30, [sp, -48]!
^
/var/folders/q1/0jsxyqlx6jl_592pvc_z3lx00000gn/T//ccLAjiBh.s:21:2: error: unknown use of instruction mnemonic without a size suffix
mov x29, sp
^
/var/folders/q1/0jsxyqlx6jl_592pvc_z3lx00000gn/T//ccLAjiBh.s:25:2: error: invalid instruction mnemonic 'adrp'
adrp x2, .LANCHOR0
^~~~
This is the procedure that finally gave me a (non-working) g++ binary:
#download aarch64 gcc arm compiler source (https://developer.arm.com/tools-and-software/open-source-software/developer-tools/gnu-toolchain/gnu-a/downloads)#extract and cd into directory
./contrib/download-prerequisites
mkdir build && cd build
../configure --build=x86_64-build_apple-darwin19.5.0 --host=x86_64-build_apple-darwin19.5.0 --target=aarch64-none-elf --prefix=/Users/nick/Downloads/aarch64/aarch64-none-elf --with-local-prefix=/Users/nick/Downloads/aarch64/aarch64-none-elf/aarch64-none-elf --with-gnu-as --with-gnu-ld --disable-libstdcxx --without-headers --with-newlib --enable-threads=no --disable-shared --disable-__cxa_atexit --disable-libffi --disable-libgomp --disable-libmudflap --disable-libmpx --disable-libssp --disable-libquadmath --disable-libquadmath-support --enable-lto --enable-target-optspace --disable-nls --disable-multilib --enable-languages=c,c++
make -j 8
stdlibc++-v3 gave me a lot of problems when compiling that I could not figure out, so I tried to compile g++ without libsdtcxx. I figure that is the root cause of all subsequent problems. Anyway, I wanted to share my workflow just in case someone else wants to have a stab at it.
For the record, compiling the c compiler with the configure parameters below works, it's the c++ compiler that's giving me a hard time.
download gcc arm source
extract
./contrib/download-prerequisites
mkdir build && cd build
../configure --build=x86_64-build_apple-darwin19.5.0 --host=x86_64-build_apple-darwin19.5.0 --target=aarch64-none-elf --prefix=/Users/nick/Downloads/aarch64/aarch64-none-elf --with-local-prefix=/Users/nick/Downloads/aarch64/aarch64-none-elf/aarch64-none-elf --without-headers --with-newlib --enable-threads=no --disable-shared --disable-__cxa_atexit --disable-libgomp --disable-libmudflap --disable-libmpx --disable-libssp --disable-libquadmath --disable-libquadmath-support --enable-lto --enable-target-optspace --disable-nls --disable-multilib --enable-languages=c
make -j 8
make install-strip
Yesssss, making some good progress here! It's been a wild adventure already. I am very grateful Patrick already did the heavy lifting on the Audio Injector Octo card in his vGuitar project, because it took me a while to get it to work properly.
When I started to use some of his code I knew zero about i2s or DMA. Niks, nada, noppes. And to be honest, I still don't fully understand it, but after staring at the code for hours, I now have at least superficial understanding of how it works, and can go on to the next steps to be taken. Right now only 2 output channels are working using a modified version of circle's built in i2s output code. Patrick actually rewrote it into something he could use with the Teensy audio library. There is a chance I might make something similar myself in the future when I want to get audio input working. The reason why it doesn't work out of the box with circle's code is because the Audio Injector should be the i2s master, and the circle code is made for the Pi to be the i2s master. It took some poking around to get things working. But like I said, thanks to Patrick for figuring out the CS42448 initialisation part, and the part to make the pi accept a master clock from the i2s bus.
The second reason is that the Octo actually uses something called TDM, that is to my understanding taking abuse of the i2s protocol to transfer more than 2 channels. Apparantly i2s was only designed for transporting two channels simultaneously. TDM is a way to get multiple channels working over the i2s bus.
The entire project now seems very promising since it left the theoretical concept stage. I've been giving the gui some thought as well, and think I'm going to create it with two types of input in mind: the touchscreen, and a rotary encoder (and other buttons). You should be able to do everything with the touchscreen, but also be able to connect a rotary encoder like on the mpc1000 and s6000 to change values fast.
I'm working as a freelance video editor for 6 days a week in August, so development for this project might be a bit slowed down during that time because I will be mentally drained when I get home in the evening, and imagine I'll need my Sunday to recuperate from screen fatigue.
Today i received the audio injector octo 8 soundcards. I'm looking forward to figure out how to get audio input and output. I'll have a look at Patrick's vGuitar rig project code, because he already figured out a lot of the details.
In the mean time I have been busy with creating the ui code. Let's get something straight about that. I just absolutely love 1 bit guis, and both the mpc1000 and s6000 have 1 bit guis because of their use of graphic 1 bit displays. So, I'm creating a 1 bit gui for this project. I hear you thinking: this guy... I know, tastes differ, and you may not like the pixellated look of 1 bit displays, but with those limitations, a minimalistic design comes with it and thats the way I like it. Further more, if at one point we discover that we dont like touchscreens, we can always replace it with a 1 bit spi/i2c display with minimal change. Also, since this is an open source project, you are free to fork it, and create a gui to your liking! While looking at the guis of the mpc1000 and the s6000, I stumbled upon the gui of the mpc4000. It's like a mix between the mpc1000 and s6000, and it's a great source to base the gui on.
The source of the first working version is going on the gitlab repo soon. Stay tuned!
Nick Verlinden
