Project Logs

Collapse

• Scope Creep and Memory Management

  Gordon • 04/17/2025 at 00:09 • 0 comments

  After continuously having issues with any network requests failing 20-80% of the time, I figured out that the issue was running out of memory. I didn't suspect this was the issue, as the biggest request I would be doing is only 1-2Kb right?

  Well after checking micropython's "mem_info()" output, I realized that the heap was entirely fragmented to the point there was no contiguous space for a 1-2Kb chunk of data to be stored.

  So I set to work to figure out how to fix this, and that's when the scope creep was fully realized. Currently the program has these classes:

  1. MiniPico: Contains device functions and any system functions, oled output, etc
  2. SecondCore: Handles playing sounds over buzzer, reading key presses, and timers
  3. KeyPad: Handles the logic for what characters to return based on what buttons are pressed
  4. Console: The console
  5. Interpreter: The interpreter program
  6. Screensaver: a simple screensaver program

  I tried to figure out how to have MiniPico as one file and have these other programs in other files then import them, but as each of these classes call functions in other classes, I was not able to figure that out. It wouldn't work. I need to research that more, it would be a good thing to know how to do.

  On to the fragmentation issue. At first I started wondering if it was almost entirely because of all the string operations I had going on. I think this was a huge part of it. This lead me onto a path of playing around with gc to see what is using memory and when. It was very informative, and i'll cover that more in a bit.

  Back to the string thing... This software is mainly text based software. I am creating a console for this device, so I have to use strings right? Maybe not, I am not opposed to using strings, but I realized I could have fixed size byte arrays to store the buffers for the console and screen text. This will let me reuse the same memory space for that. Another thing I can do, instead of having string operations happening in each class, and string data stored in each class, put it all in one class or object at the beginning of the file so it's all in one chuck and not scattered throughout the entire heap. I think this was the main issue, but not entirely sure.

  Once I realized my program needed to be rewritten, I was on the verge of switching to and learning the C SDK. Mainly because you have to have complete control over memory in C, so you know how much memory is allocated and where (generally). I love C programming and want to learn more, and want to try C on the Pico, but getting the environment setup and rolling is something I haven't visited yet. I did install VSCode and the Pico Extension, and was about to start experimenting (I will soon), but then I had the thought. Why just switch to C when this is a great opportunity for learning more about how micropython manages memory, and it will be a good challenge to try to write a large program that doesn't have memory issues.

  On to my findings while experimenting with micropythons garbage collector: Here is the whole code that I was playing around with.

  # test micropython program for experimenting with memory usage,
  # and byte arrays. On P Pico W
  
  from micropython import mem_info
  import gc, random, time
  
  ROWS = 32 # Integers use 8 bytes each
  COLS = 16
  PERCENT_100 = 100
  PERCENT_25 = 25
  PERCENT_50 = 50
  buf = bytearray(ROWS * COLS) #main buffer byte array uses ~550 bytes
  out_buf = bytearray(16) #output buffer
  
  gc.enable()
  MAX_MEM = gc.mem_free() #set upper memory threshold at current level
  gc.threshold(int(MAX_MEM / PERCENT_100 * PERCENT_25)) #collect when memory is 25% full
  
  
  def get(buf, row, col):
      return buf[row * COLS + col]
  
  def set(buf, row, col, value):
      buf[row * COLS + col] = value
  
  def free_mem():
      #mem = ""
      #mem += 'mem: '
      #mem += f'{round(gc.mem_free() / MAX_MEM * 100)}'
      #mem += '%'
      #return mem
      #return round(gc.mem_free() / MAX_MEM * 100)
      #return 128 * random.randint(0, 64) + random.randint(0,64)
      return gc.mem_free()
      
  def main():
      
   start_time...

  Read more »

• A Little Progress

  Gordon • 04/14/2025 at 21:50 • 0 comments

  Timers

  I think the amount I jump around working on one feature to another is kind of a good representation on how my mind works and how I work on something then quickly switch to another thing. That's why I actually like having multiple ongoing projects at the same time. I really don't have time to work on any projects right now but end up working on them when procrastinating other duties I have. I need to get my life together.

  I have been working on adding timers to the MiniPico.

  There are a few different "sleep" modes for the device, both modes shut off the OLED first:

  1. Sleep Mode 1, goes into lowest power mode and only can be woken up by pressing "ESC" button (top left button)
  2. Sleep Mode 2, when user specifies a sleep amount in minutes eg: "s.z10" (10 min), device will sleep for that amount of time. Due to the time stopping counting when the lowest amount of power sleep mode is used, this sleep mode sets frequency to 20mhz, and does nothing until time is up or ESC button pressed. This mode does not save a ton of power, but allows timers to be used

  A timer can be called by "s.tt<n>", with n being the number of minutes we want to set the timer for. The second core handles the timers, and if a timer reaches it's end, it beeps twice.

  If the user doesn't press a button for a set amount of time, the device will enter a sleep mode. If there are any timers set, the device will enter Sleep Mode 2. If no timers are set, it will enter Sleep Mode 1.

  If timers are set, the timer that is next in line (time wise) will be the amount of time the device sleeps for.

  This took me a while to figure out, but it's working great now. As the device is just using Pico W's internal timing and there is no real RTC, I'm sure these timers are off by a few seconds, but that's perfectly acceptable when just setting a timer for a few minutes.

  I want to add alarms as well, for example every day at 6am. But this would require the use of an external RTC, so that's off the table for this device.

  Post

  I also wanted to find a way to send a "tweet" (I don't know what they call them anymore now that it's not twitter) to my X profile. I tried messing around with the X API directly, but to no avail. Then it appears that most people are using IFTTT, but now they charge for webhooks. So that's a no go.

  So my solution was to create POST, and host it on my website. Basically this is a "chat" that if someone has the secret key, they can post a message to.

  If connected to WiFi, the MiniPico is able to read all messages using the command "s.g". I had to implement text wrapping to OLED output, as well as add the ability to page up/down the console output history. Currently there are 21 lines that fit into the history, or 3 pages. That's enough to read the whole list of 8 posts.

  On the server side, it's a text file that every time a post is written to, it bumps the last one off, and adds the new one. Pretty simple really.

  The secret key is hardcoded into the MiniPico's firmware, so to make a post all that is needed is to write "s.p <message goes here>". I need to implement the ability to write lines longer than 16 characters, as this allows for a maximum of 12 characters to be posted.

  Often when doing urequests stuff on Pico W, I get the "[Errno 12] ENOMEM" error... So I have to catch them with a "try/except" to allow user to try again. I need to research this more.

• Space Roguelike Game/UI

  Gordon • 04/01/2025 at 16:58 • 0 comments

  I have had little time to work on this project, but instead of continuing with the interpreter/device code, I started making a little roguelike game the device.

  The basic idea for the game is the player has a ship, and can travel between several different planets. Possibly do some trading.

  Landing on a planet will generate a map, essentially an 8 level dungeon with enemies and a boss at end level.

  After making a playable map, as well as some basic item interaction I got the idea that I could use the game itsel as the UI for this device. While the player is in the ship, they can access the console screen, or check device battery levels or other device settinngs.

  Travel between planets will use constant accelleration. The old "Turn and Burn" procedure.

  All tiles are 8x8, and as the device screen is monochrome, the sprites are stored in bytearrays, so each individual sprite is only using 8 bytes of memory. This will allow many available sprites. The goal for the game itself is to have each planet have its own tileset, enemies, items, and resources.

  I also want to look into prcedurally generating different types of sprites, specifically terrain types and enemies.

• Small Updates

  Gordon • 02/10/2025 at 19:52 • 0 comments

  Haven't done much on projects since back from my trip. But I did fix some issues with the wifi, so it is much more stable now.

  I also made it so device will enter deepsleep after 1 minute of no button presses. The deep sleep state that it goes into must use less power than I previously thought, because it was in this state for nearly one entire day and barely used any battery... maybe 5%. I haven't done many tests, but the main purpose of this state is if the user forgets to turn it off the battery won't just drain uselessly.

  Unrelated to this project: My brother I was visiting is an electrician, and one of the jobs we went to there had been a lightning strike near the house. We swapped out some outlets and switches at the site, as well as an automatic yard sprinkler control box. I was curious what had been damaged in the box. And upon opening it saw many exploded capacitors and transistors, as well as traces that had been completely blown off the board. I got a couple pictures and may make a page post with my findings as it was interesting.

  I am really wishing I would have started this project in C instead of python. But I will continue work on it until the interpreter and device code is finished in python before moving on to C. My goal is to make the GRK code compatible with the python and C implementations of the interpreter... The C version will also have a PC version, and can run a GRK program file by doing something like "./grk test.grk", or just running "./grk" or "grk.exe" if on PC, which will load the GRK console.

  I am not too familiar with Windows command line options, but when using a linux environment I like creating alias's for my own programs to make things simpler. 

• Video of Project Current State

  Gordon • 01/21/2025 at 05:57 • 0 comments

• Going Forward

  Gordon • 01/20/2025 at 18:07 • 0 comments

  I will be on a trip til the end of the month so will not be working on any projects during that time. But I have some thoughts on the future of this project and the interpreter as a whole.

  I am going to rename the language to just GRK. The initial version will be written in python, then I plan on making a version in C. Both implimentations will be available on my github in the same repository.

  I think micropython is fine for most cases on embedded devices such as Pico W if wanting to make something quick, or it's a small project. But as python is very inefficient on memory, with a larger project it is not very good. I am already having memory issues I need to sort out with what I have running  and it's really hard to track down where it's happening. So far it is almost entirely to do with using the urequests library for fetching data from the internet. I think I can optimize it in python, but if I were using C I doubt I'd have run into this issue yet as the total available memory would be much greater before even fetching the html request.

  When I wrote TGRK in C for attiny85, if I remember correctly I think the entire interpreter only used about 100-150 bytes of memory (not counting program memory, which in TGRK is set at 127 bytes). Granted TGRK does have a fraction of the features GRK will, but with this project, for it to work reliably I need to be able to manage the memory myself.

  A simple example is an integer variable for setting different modes. Let's say we have modes 0-3. In python this will automatically be several bytes. In C we can make sure it is only 1 byte by setting type as uint8_t. If we want 8 flags that can be 1 or 8, we can save that in a single byte and manipulate the bits in that byte. I need to lookup how many bytes micropython uses for a boolean. it has to be more than 1 considering how python stores variables.

  I have not used the C SDK for Pico or Pico W, so I will be setting that up and driving into that after the python implementation is finished.

• Browser Based Programming

  Gordon • 01/17/2025 at 22:58 • 0 comments

  The title is just about one feature I've added, but there are a lot of new features, so many in fact I may forget about some.

  I have been having so much fun adding device features I have still not touched the interpreter.

  I will not list interpreter commands here as there are no new ones. But I do intend to make many or all of these system commands accessible from the interpreter.

  A picture of the browser tool:

  Below is the list of all system commands up to this point.

  Pico PicoW Device Specific Commands:

  s.w: If this command is run like this, and no ssid and password is currently set, it will scan for networks, then check if there are any saved networks that are available. if so it will automatically connect. If there is already a ssid and password set, it will try that first.

  • s.ws: Scans for available wifi networks
  • s.wc<n>: Connect to network "n" from scanned list. Acts the same as "s.w" if no argument provided.
  • s.wl: Display the scanned list of networks on screen
  • s.wx: Turn off wifi module
  • s.wa: Create a WiFi AP, prompts user for ssid and password.

  s.s: Creates a webserver, outputs IP address of server to screen. Webserver has a text area field that programs can be written into. If server receives program data, it replaces all the html characters with appropriate characters, and outputs it to screen. As of now it automatically appends to programs list. I will make this optional in future. As well add option for executing program immediately. Entering "end" into browser text area will shut down server.

  s.g: Currently running this as is will connect to "https://g0730n.com" and output "success" if it receives a 200 code back.

  • s.g<url>: Currently works the same as above, except any url can be fetched: "s.ghackaday.io"
  • s.gt: Fetches time from NTP server, updates system clock.
  • s.gw: Fetches weather data from API, saves data to text file as it is good for 48 hrs.

  s.t: Displays current date and time

  • s.ta: Non functional at moment, but will be used for alarms and timers.

  s.c: Displays weather graph of saved weather data.

  s.r: Not sure if is needed... Shuts down second core that scans keys, does a machine reset. Was having issues with connecting in Thonny so this was an attempt to deal with that. I did find a method that worked. So I dont think this is needed. But I will use this in future to do a reboot.

  s.z: Device sleep command, called as is without arguments, device will turn off oled, wifi, and enter deepsleep indefinitely. Pressing ESC button wakes device. RTC time does not increment in this mode. I have not tested yet, but this mode should only use 1-2mA of power. 

  • s.z<n>: Turns off screen and WiFi as above, calls "time.sleep_ms(n)", n being 0-70 minutes. Will be used in alarm and timer modes in future. This method of sleep does not save a ton of power, but better than nothing. Drops machine.freq() to 20mhz before entering this sleep mode, then goes back to whatever it was before after finishing the sleep cycle.

  s.f<n>: Set the machine frequency. n being 20-250(mhz)

• Experimenting With API's

  Gordon • 01/15/2025 at 16:14 • 0 comments

  Before I go back to working on the interpreter I decided to add some more system features.

  "s.g" can be used to fetch a URL if connected to the internet.

  "s.s" Will create a server and allow client to load a web page either over Pico AP or through LAN depending on Pico W's connection state. The web page has a single text area for writing a program and uploading to device. WIP

  "s.gw" will call weather API, get the next 48 hours of data. 48 data points of Temperature, Humidity, Cloud Cover, and Chance of Precipitation. The API I am using can be fined tuned for many different weather data points but these were what I thought I would find most useful.

  "s.c" will show a graph of this weather data:

  In the photo above the randomly scattered data points are from the cloud cover plot line. the lower it is on the screen the sunnier it is. The numbers on left hand side are for % of (clouds, humidity, chance of precipitation) or temperature in Fahrenheit.

  Connection to internet over wifi, on initial connection RTC time will be synced with NTP. This can then be retrieved by the command "s.t".

• System Tools

  Gordon • 01/15/2025 at 01:55 • 0 comments

  I discovered that I didn't lose as much progress as I thought.

  Currently every feature that is available in the PC developement version is available and working on device.

  I am trying to figure out a good way to keep the interpreter separate from the devices code. In the sense that routines for handling the devices wifi connection or printing a line out to the OLED should be accessible from the interpreter while running on the device, but not while running it on PC.

  I decided to create an option from the device commandline to type 

  "s.<system command>"

  Currently all I have implimented is a command line utility on device that can:

  1. [ s.wa] Create wifi AP, using a hard coded ssid and password or a new one
  2. [ s.ws ] scan for wifi networks. running this cmd saves a list of available networks
  3. [ s.wl ] view the list of networks saved in last scan
  4. [ s.wc<n> ] connect to the network of number specified from networks in list, this will prompt for a password. if command is run without a number, it will try to connect to the last saved network running [ s.w ] will do the same thing as   [ s.wc]
  5. [ s.wx ] turns off wifi, whether AP or Client, it goes off. I have it set up so only AP or client can be run at same time.

  I will later impliment a save file on flash that will have different saved settings including saved networks and passwords. when running [ s.w ] in that case it will automatically scan and see if there is a saved network that matches, then connect.

  I have also created a new function specific to the Pico Picow for easier printing of lines and getting input from anywhere in the devices code. While running the interpreter, the device is basically always stuck in an input mode. But I realized these system tools will need the ability for input that can be accessed out of the interpreters command line.

  I really want to get back to adding functions for the interpreter, but wanted to make sure 100% that it was all going to work on the device okay before I went any further. So far so good. I have a better backup system I am using right now too.

• Version Control

  Gordon • 01/13/2025 at 17:00 • 0 comments

  I stayed up late last night working on the interpreter. I made a lot of really good progress. But this is real life and often times there is good news and bad news.

  I moved the entire interpreter into a class. I am not sure why I didnt start it out this way, but I realized it was inevitable and the longer I went without doing it the more tedious it would be.

  I spent a lot of time working on error handling, and was able to impliment and catch syntax errors that would before freeze the interpreter.

  I then decided to make the first port of the PC developement interpreter to the Pico Picow. This was also going great. It was exciting to finally be able to write commands on the devices console. I added a larger command history on the device, from it remembering the last command to remembering the last 8.

  And then as I was testing it on the Pico device, I went to save it again and somehow overwrite the last several hours of work. I know I was tired but this was unacceptable. I need to use a better version control than just saving multiple copies at random times. I did not lose all my work, just from wherenI started porting it to the device.

  So, these features I speak of exist, but I will have to re write them. Once I get back to where I was with the code, I will make a video as that was a good level of functionality to demonstrate the device.

View all 13 project logs