Live-coding an Open-Source Pico Emulator from Scratch
To make the experience fit your profile, pick a username and tell us what interests you.
We found and based on your interests.
rp2040_datasheet.pdfThe RP2040 datasheet, where we'll find all the specific details about the MCU core and peripherals.Adobe Portable Document Format - 29.77 MB - 01/25/2021 at 18:44 |
|
|
DDI0419E_armv6m_arm.pdfARM® v6-M Architecture Reference Manual. It'll be our main reference for implementing the emulator!Adobe Portable Document Format - 2.16 MB - 01/25/2021 at 18:37 |
|
|
getting_started_with_pico.pdfPico Getting Started guide. Check out Chapter 2 for setting up the build environment. We'll use it extensively while working on the emulator.Adobe Portable Document Format - 36.28 MB - 01/25/2021 at 18:36 |
|
|
This week we started working on the PIO implementation:
We also held a spontaneous stream where we created a test suite for the PIO instruction set, running it against the silicone rp2040:
We'll continue hacking on the PIO this Tuesday.
In the last live stream episode, we implemented GPIO (+ Interrupts). But we also figured out that we're just a small step away from running MicroPython in the emulator.
And so, yesterday I decided to go on a spontaneous live stream, trying to implement UART RX in order to close the gap and get MicroPython running!
Spoiler: We did it! You can now run MicroPython with RP2040js. I also uploaded a quick proof-of-concept to Wokwi. It works well, though it's not very polished at the moment:
https://wokwi.com/arduino/projects/299738191162769933
After opening the link, don't click the green "Play" button. Instead, follow the instructions in the comment at the top of the sketch file. Enjoy!!
We spent the last coding live stream implementing the Timer Alarms for the Raspberry Pi Pico:
We also changed our Peripheral abstraction to handle Atomic I/O Register Writes and started working on the GPIO peripheral (refactoring SIO along the way).
To be continued....
gdbdiff is a fast and reliable way to check our instruction set implementation. It runs the same piece of code on both the physical Pi Pico (the silicone) and the emulated Pi Pico, instruction by instruction, and compares the CPU state after each instruction.
We held a bug squashing session yesterday and spotted several instruction bugs thanks to gdbdiff. The gdbdiff part starts at hour 1:25:00 into the stream—
gdbdiff works by using the GDB Remote Serial Protocol to debug the Pi Pico and the emulator. It has a client implementation of the protocol which connects to the RP2040 silicone through OpenOCD, and to the RP2040 emulator using the gdbserver stub.
Thanks to gdbdiff, we were able to find and fix several bugs, mostly related to the Cortex-M0+ flags in the APSR register, and also an edge case in LSRS:
The last issue in the list also caused a funny behavior of printf() when printing numbers from the Arduino Mbed OS core.
Using gdbdiff proved to be a very efficient way to spot tricky bugs. Previous attempts to spot such bugs required hours of laborious debugging, and now we have a tool which automates the process.
At this point, the emulator's instruction set is mostly ready. There are still some multi-core and interrupt-related instructions that we need to implement (e.g. SEV, WFI, WFE), but they shouldn't normally affect the code flow.
We'll keep hacking on the emulator this Tuesday, adding Timer Alarms, GPIO, and maybe also the SIO divider. See you then!
Whoa, this week had three(!) spontaneous live-streams, in addition to the usual weekly stream. It all started when we tried to get the official Arudino Core for the Pi Pico to blink a virtual LED:
We started working on the implementation of SysTick, a timer built into the ARM Cortex Core, and ran into problems. So we did another live troubleshooting session:
As we couldn't really figure out the problem, we decided to start tidying up the code and implementing all the missing instructions, in another spontaneous session:
Finally, following a suggestion from Mike Wright on the YouTube channel and a GitHub discussion with Valerio, we decide to verify our unit tests against the actual RP2040 silicone. We also set up ESLint, discovering a few small but significant bugs in our code:
Well, it's definitely been a crazy week. See you in the next stream today!
In one of the recent live streams, we tried to debug a program compiled with the official Arduino Core for the Pi Pico.
However, debugging it proved difficult: the Arduino Core produces programs without debugging info. In addition, it's based on Mbed OS, an embedded operating system for ARM Cortex-M processors. Mbed OS adds another layer of complexity: it manages tasks and threads, and does a lot of work behind the scenes even for a simple Blink program.
Therefore, I decided to compile the Arduino Core for Pi Pico from scratch, and this this with debugging symbols. This log explains how I did it. If you want to follow it, you need an Ubuntu or Debian-based Linux system (for Windows users, WSL is also fine).
Install the Mbed OS CLI. For Linux:
sudo apt install python3 python3-pip git mercurial
python3 -m pip install mbed-cli
For other environments, check out the Mbed OS CLI documentation.
In addition, the Arduino Mbed OS core requires jq. Install it too:
sudo apt install jq
git clone https://github.com/arduino/mbed-os arduino-mbed-os
git clone https://github.com/arduino/ArduinoCore-mbed
Note how we clone mbed-os from Arduino's GitHub. They have their own fork with patches specific for the Pi Pico. At the time of writing, they still haven't merged these patches upstream.
Apply the following patch to the ArduinoCore-mbed. The patch enables debug build:
diff --git a/mbed-os-to-arduino b/mbed-os-to-arduino index 902b0c5d..921c39a8 100755 --- a/mbed-os-to-arduino +++ b/mbed-os-to-arduino @@ -130,6 +130,9 @@ mbed_compile () { PROFILE_FLAG=--profile="${PROFILE}" fi export PROFILE=-${PROFILE^^} + else + export PROFILE="-DEBUG" + PROFILE_FLAG="--profile=debug" fi
cd arduino-mbed-os
git checkout rp2040
mbed deploy
cd ../ArduinoCore-mbed
./mbed-os-to-arduino -r "`pwd`/../arduino-mbed-os" RASPBERRY_PI_PICO:RASPBERRY_PI_PICO
You may be able to skip mbed deploy
- I not sure if it's actually needed.
If everything went well, you'll find the compiled library under ArduinoCore-mbed/variants/RASPBERRY_PI_PICO/libs/libmbed.a. The debug symbols make it pretty heavy - expect a 135MB file.
You can copy this file over your existing libmbed.a in the Arduino directory, e.g.:
cp ArduinoCore-mbed/variants/RASPBERRY_PI_PICO/libs/libmbed.a ~/.arduino15/packages/arduino/hardware/mbed_rp2040/2.0.0/variants/RASPBERRY_PI_PICO/libs/libmbed.a
The target path may vary depending on your system configuration and the version of the mbed_rp2040 core you have installed.
Happy debugging! ;-)
We spent the last two episodes working on the Nested Vectored Interrupt Controller (NVIC), adding some more instructions (UXTH, MULS, REV), and working on the special register implementation (MRS / MSR / CSPID i / CPSIE i) instructions.
We got Hello Serial to work, and started looking at running the Arduino Core for Pi Pico, based on Mbed OS, in the emulator.
Along the way we also found how to make GDB much faster by disabling Nagle's algorithm in our gdbserver. Now debugging the emulator is so much more delightful!
Interrupts, part 1:
Interrupts, part 2:
In the next episode, we will implement the SVC instruction, look at an interesting bug in our MULS implementation, and try to get the Arduino Core "blink" to work. See you tomorrow!
Last week we released an alpha version of RP2040js to NPM, the JavaScript package manager. We spent the first hour of the stream tidying up the code, setting up the tooling and preparing the release.
Then, in a sharp turn, we went back to working on the code: refactoring the peripherals, implementing narrow I/O register writes, adding a missing variant of the CMP instruction, and even creating the Timer peripheral.
Our new goal is to get hello_serial to run. Unlike hello_uart, it uses a set of high-level APIs, including printf() and sleep_ms(). That's also the reason we needed the Timer:
In the next episode, we'll keep working on the Timer, and probably also implement the Nested Vectored Interrupt Controller (NVIC), to let us fire interrupts from our virtual peripherals.
This week we managed to get a Pi Pico program to run start to end in our emulator!
The happy moment happened just as the stream was about to conclude, about 2:29 hours into the stream. You can watch the full recording here:
Or, if you want to try running hello_uart yourself and replicate our success, you can clone the repo and run the code:
git clone https://github.com/wokwi/rp2040js
npm install
npm start 2>&1 | grep UART
The "grep" part takes care of hiding all the debug prints we have in our code... 😜
Alternatively, you can also run it in the Gitpod cloud. After launching the workspace simply copy the two npm commands above and paste them into the terminal. Enjoy!
Next week, we are going to tidy up the code and publish a first alpha release of rp2040js to npm. So if you are reading this after April 6th and still want to run the hello_uart example, make sure to git checkout 175f616.
So far we spent about 25 hours of coding to unlock our first milestone: running a basic program from start to end in the emulator! 🔓
The next live stream happens on Tuesday, and you'll be able to watch it here.
During our last live-hacking session, we discovered the compiler optimizations were severely limiting our ability to debug the code in the emulator. Function inlining, code rearrangement and tail call optimizations made debugging with GDB much more challenging.
After digging a bit in the Pico SDK's build system, we learned about a flag that disables all the compiler optimizations, PICO_DEOPTIMIZED_DEBUG
.
Here's the complete sequence of commands for building the Pico examples in debug mode and without any compiler optimizations:
cd ~/pico/pico-examples/ rm -rf build mkdir build cd build export PICO_SDK_PATH=../../pico-sdk cmake -DCMAKE_BUILD_TYPE=Debug -DPICO_DEOPTIMIZED_DEBUG=1 .. make -j4
It is also possible to build just a specific example, by running the make -j4
command in a subdirectory of the build directory. e.g., to only build hello_uart, we'll first change the directory to uart/hello_uart
:
cd uart/hello_uart make -j4
Create an account to leave a comment. Already have an account? Log In.
Thanks Tony! I have just updated the README and fixed the instructions.
I dream to run Palm Os open source
http://dmitry.gr/?r=05.Projects&proj=27.%20rePalm
or
https://pmig96.wordpress.com/2020/09/26/palmos-on-x86-api-porting/
Interesting project! Which MCU did the Palm device have?
That's really cool. I always wondered what Palm OS would be like on modern hardware now that we have stuff like e-ink and memory lcd. I wonder if the battery backed ram stuff could still be done by anything. It's been too long ago but I remember the Palm OS stuff had everything as a database I think. There were some cool ideas there anyways.
Become a member to follow this project and never miss any updates
Hi, wokwi, nice work. I installed again your project, and the micropython running instructions were incorrect in the README, it needs uf2 file rp2-pico-20210618-v1.16.uf2, not hex file. After getting that it runs.