Close

GUI

A project log for More accurate thermal imaging cameras

A pair of calibrated thermal imaging cameras with ambient correction based around the Tiny1C thermal imaging sensor.

dan-julioDan Julio 11/20/2024 at 22:452 Comments

(pretty pictures below if this is TL;DR)

I'm using LVGL for the iCam and iCamMini Web GUIs. The video output uses a much simpler set of 8-bit monochrome drawing routines since LVGL doesn't really support 8-bit monochrome.   I chose LVGL for the web interface because I don't really know traditional web programming but am experienced with LVGL and can use emscripten to compile a C program into WebAssembly that would run in any modern browser.  The LVGL people already use emscripten to publish live demos in their HTML documentation so it was easy to figure out how to build.  The result looks slightly different than traditional web page layouts but it works.

The part I'm proud of is creating a view/model-controller paradigm that let me write essentially one set of GUI code works both on the embedded (SPI-accessed) 3.5" gCore LCD and also in a browser.  When running in the browser it can resize and react to events like rotation of a mobile device to change its layout dynamically.  The connection between the GUI view and model-controller is done through a set/get/response API that maps to direct function calls when the GUI is running on the same platform as the model-controller and serialization/de-serialization through a websocket when the GUI is running in the browser.  For example the GUI might issue a GET EMISSIVITY command to update the emissivity control panel.  This ends up executing a get_emissivity_handler() subroutine.  The subroutine is executed directly by the API on the combined platform and as the result of parsing an incoming websocket packet on the remote platform. 

I do take a few performance short-cuts.  For example the actual thermal image is a 256 x 192 x 8-bit scaled raw image chunk of data.  All 49,152 bytes (+ some additional data) have to get sent over the websocket by iCamMini.  But on the iCam platform I only pass a 4-byte pointer to a shared data structure instead of copying the data.

I say the GUI can automatically adjust to the screen but there are limits.  In all cases during my layout I had to make sure it would fit on the relatively constrained iCam 480x320 pixel LCD.  For performance reasons I picked fixed 1.5x or 2.0x magnification of the image (so I could use some very optimized bi-linear interpolation code)  and this sets how big the display can be on a desktop browser.  In other cases I adjust limits so the various screens look good on a mobile phone.  I have yet to do testing on a tablet so it's possible further optimizations might be necessary.

When running on an iCamMini, the code starts the Espressif web server which serves a single large html page to the first connected client.  This page contains a very small amount of HTML and CSS and the big WebAssembly binary (> 400kB)  that runs when the page loads.  It initiates a websocket connection with the server and from then on all commands and data pass through that.  The web LVGL drivers render to a javascript canvas using SDL.

The WebAssembly code is its own program and can run even when the iCam is disconnected from it for some reason.  For this reason I am attempting to catch various disconnect scenarios or have iCamMini inform the GUI when it is powering down so the GUI can display a default page indicating it's disconnected (and has a reconnect button).

All C code is contained in a single project structured for the Espressif IDF build environment.  The majority of the code is shared between the two cameras with only some custom device-specific code.  I have a script which sets specific sdkconfig files for each camera type and the linker is configured to only include necessary code.  There is another directory that is ignored by the IDF tools but is where I build the emscripten code.   It contains its own main.c file which is the WebAssembly LVGL GUI program.  There are links back to the IDF components directories holding the LVGL GUI code.  In this way the same GUI source is built for two different platforms.  All-in-all the code base builds 3 binaries:

  1. iCamMini ESP32 code
  2. iCamMini WebAssembly code (which is included in the ESP32 code ultimately)
  3. iCam ESP32 code

But enough of this wall of text.  Here are some actual screens.  The main screen can render in a landscape or portrait orientation.

The settings page is a scrollable container of panels, one panel for each control or related set of controls (which are drawn on another screen).  Navigation between settings screens is done via the typical ">" and "<" buttons.

Finally there is a simple File Browser screen that allows viewing the JPG images saved on the Micro-SD card.  Here shown as a mobile screen shot and on iCam.

I'm following the traditional digital camera DCIM directory structure with a set of folders (NNNICAMF) each containing up to 100 images (ICAM_NNNN.JPG).  I limit the number of files in a folder to help speed the underlying filesystem code running on the ESP32.  An astute observer may wonder why, then, I have a file with the number 30 in the 2nd folder.  This is because during testing I varied the number of files to test some boundary conditions in the code that generates the filesystem catalog.

Discussions

jian.jiang wrote 11/21/2024 at 01:03 point

gorgeous

  Are you sure? yes | no

Dan Julio wrote 11/21/2024 at 15:48 point

Thank you.  LVGL makes pretty good GUIs I think.

  Are you sure? yes | no