Lepton 3.5 Thermal Imaging Camera

Just finished a video showing how to assemble a tCam.

Right now you have to buy parts separately (gCore from Group Gets or Tindie), tCam-Mini and the Lepton 3.5 from Group Gets and source the misc parts but hopefully soon there will be a full kit available from Group Gets.

I wrote a simple utility program that runs on 64-bit Linux x86, OS X or Windows to make it easy to load the most recent firmware (downloaded from my website) for either tCam (gCore hardware) or tCam-Mini/tCam-POE.  I did this primarily to make it easy to program a gCore for those people who are building tCams but it can also be used to load the most current tCamMini firmware as well.

It simply requires connecting the camera's USB serial port to the computer and programming.

It can be downloaded from my website.  Look for the tCam Family Serial updater section.

The application itself is pretty simple xojo.  It's able to program the ESP32 on the different platforms because it includes compiled versions (for each platform) of the Espressif esptool.py python program.  I just use xojo's shell object to run the binary version with the selected serial port and paths to the downloaded firmware binary files.

I was led to this idea when I found that the Espressif ESP32 Arduino package included compiled versions for Windows and Mac OS X.  I dug around their github repositories until I found a script that uses pyinstaller to compile the python esptool.py.  I only had to create a binary for Linux x86 which I did with the following command after installing pyserial and pyinstaller.

pyinstaller -F --add-data="esptool/targets/stub_flasher/*.json:esptool/targets/stub_flasher/" esptool.py

This left a built executable called esptool in a dist directory.  Very cool.

I finally had a moment to add the tCam-POE hardware design to the tCam github repository.   tCam-POE is basically tCam-Mini but with the ability to connect via ethernet instead of Wifi and also supports IEEE802.3af POE power.  No word yet if Group Gets is planning to build this.  Let them know if you would like to see it as a product.  :-)

gCore is going to be available through a campaign at Group Gets.  This means tCam can also finally become a reality for anyone who would like to make one.  Finally.  It has felt, at times, that I'd never get to this point.

There's a dedicated gCore github repository.  

And I moved all the ESP32 thermal imaging stuff (tCam-Mini, apps, etc) to its own github repository so that people who are interested only in that don't have to download everything else I did with the Lepton.  The tCam firmware has been uploaded there as well as instructions for use and the enclosure designs.  The firmware binaries which must be loaded onto gCore and the enclosure designs can also be directly downloaded from my website.

I realize that gCore might seem a bit pricey at the moment.  Neither I or Group Gets is making much money on this campaign.  It's really a factor of the parts cost (higher than some dev boards because gCore has a lot of high end features, and those take extra silicon), the manufacturing cost as Group Gets is building the boards with a USA contract manufacturer and the initial low volume of only 100 boards.  Even so I think it allows building a full feature, open-source thermal imaging camera for a reasonable price.  Over time I am working to bring this price down.  That will be discussed on the gCore project page.

Hardware designs for gCore and tCam-POE have gone to Group Gets for quoting and buying.  I've noticed Mouser has a lot more parts recently so maybe the great partageddon is finally showing signs of easing.

I also just pushed a bunch of new code up to github and my website.

tCam-Mini Firmware Revision 3.0

Big changes so this code base can also support tCam-POE.  I also added a mDNS (bonjour) responder so that the IP addresses of cameras on a network can be found by the various applications.  Rounding this release out were a few bug fixes plus I changed the internal flash access to use 80 MHz QIO accesses in preparation for some future firmware enhancements.

Sadly the day after I made the release I found another minor bug.   A camera that has had its WiFi reconfigured to use STA mode with a static IP address may not correctly restart the WiFi stack until the next reboot.  Easy fix that will be rolled out in a bit but odd that I didn't see this before.

A very kind soul got the previous firmware to compile on Espressif IDF 4.4.1 (instead of my ancient 4.0.2) and sent a pull request so I'll be trying to integrate his work for the next release too.

Desktop Application 3.0.0

A whole bunch of changes so I'm just copying the release notes here.

1. New IsoTherm palette
2. Added ability to change palettes by clicking the top or bottom of the color bar
3. Added support to identify new Ethernet camera model
4. Added Camera Info window (opened by new Camera->Camera Info... menu item)
5. Application no longer displays firmware version of connected camera in main window
6. Added new Discover function for bonjour discovery of cameras from Preferences Window
7. Wifi/Network Settings tab understands Ethernet camera model and disables irrelevant controls
8. Added Help window accessed through Application->Documentation menu
9. Added ability to check for updated application automatically at startup or from Application Menu
10. Added ability to check for new tcam/tCamMini firmware versions from Camera Menu based on connected camera type
11. Enabled close button on About window
12. Main Window toolbar connect button toggles between "Connect" and "Disconnect" captions
13. Improved error message when application can't connect to camera
14. Internal changes with window management in preparation for multi-camera sessions
15. Signed Mac version with Apple developer ID

Here's a screen shot of the new mDNS camera discovery in operation (accessed from the Preferences Window where  you set the camera address).

Perhaps the neatest feature of the new version is the ability of the application to tell you if there is an updated version of itself or an updated version of the firmware for the attached camera and then download a zip file with the new code.

iOS App

The iOS app got the new IsoTherm palette as well as camera discovery.  The IsoTherm palette is great for quickly finding both the hot and cold spots in an image.

Python Driver

Last, but not least,  my friend bitreaper and I have been working on the Python driver to allow it to support the tCam-Mini Hardware Interface on a Raspberry Pi.  One of the things I've noticed over the years (and saw for myself) is that user space programs on the Pi have a hard time keeping up the the Lepton VoSPI pipeline.  Lots of users complaining in the forums about that. tCam-Mini with the python driver is a great solution to that problem.  In addition to communicating over a network socket the Python driver can now use the Pi's serial and SPI ports to communicate directly with a tCam-Mini.

My very first tCam-Mini PCB got new use with wires soldered directly to the ESP32 module for testing.  Here it is on a Raspberry Pi 4 running the streamtest_hw.py demo and a full speed streaming image.

The driver makes it easy to process data from the camera.  After setup the following two routines handle getting the radiometric data from the stream and converting it to an image to display using Tk.

def convert(img):

    dimg = base64.b64decode(img["radiometric"])

    ra = array('H', dimg)

    imgmin = 65535
    imgmax = 0
    for i in ra:
        if i < imgmin:
            imgmin = i
        if i > imgmax:
            imgmax = i
    delta = imgmax - imgmin
    a = np.zeros((120,160,3), np.uint8)
    for r in range(0, 120):
        for c in range(0, 160):
            val = int((ra[(r * 160) + c] - imgmin) * 255 / delta)
            if val > 255:
                a[r, c] = rainbow_palette[255]
            else:
                a[r, c] = rainbow_palette[val]
    return a

def update():
    if tcam.frameQueue.empty():
        evt.wait(.05)
    else:
        image = convert(tcam.get_frame())
        frame_image = Image.fromarray(image)
        frame_image = ImageTk.PhotoImage(frame_image)
        l1.config(image=frame_image)
        l1.image = frame_image
    l1.after(50, update)

I've been working on a new tCam model that helps deal with the limitations of WiFi in some situations, for example monitoring temperature inside metal cabinets or other industrial environments.  The tCam-POE design is based on the tCam-Mini design but also includes support for wired ethernet through the inclusion of an ethernet PHY chip and magnetics.  In fact it and tCam-Mini will run the same firmware starting with the upcoming version 3.0 release.  tCam-POE can connect using either 10/100 ethernet or WiFi (switch selectable at power up) and it can be powered via a USB-C 5V power source or, with the additional of a commonly available IEEE 802.3af DC-DC converter module, via power-over-ethernet.

Shown here are my two prototypes sporting Winchen and Silvertel POE DC-DC converters.

tCam-POE has the same command set as tCam-Mini with one exception.  The set_wifi command doesn't need to include WiFi-specific arguments such as SSID and Password and tCam-POE cannot be configured to serve DHCP addresses over ethernet (it still can act as an AP when running with the WiFi interface).  It 's ethernet interface either gets a DHCP served address from an external router or it is set with a static IPv4 address.  It also doesn't have the hardware interface that tCam-Mini Rev 4 PCB has.

Porting the code to support it was pretty easy as Espressif has built-in support for an ethernet interface and after initialization the same network stack can be used for either WiFi or ethernet.

Small changes are being made to the other apps so they can identify the new model number sent by tCam-POE.

All code will be put up in the next couple of weeks.

The plan is for Group Gets to build and stock these units.  Hopefully in the next 2-3 months - pending parts availability (I'm getting better about specifying parts with multiple sources...)

While I wait for Xojo to get their act together wrt Android support, a friend decided he could use my project as an excuse to refresh his Android development skills.  His work is paying off and he sent me an alpha-level APK to play with today.  For everyone who has patiently been waiting for an Android App that can talk to the tCam family of cameras, I present this tease.

If all goes well there will be a beta APK file for people to side load in a few weeks.  My friend's longer-term goal is to get the app into the Play store.

It's crazy sometimes how fast time seems to fly by.  I have been working on tCam for over two years now in fits and starts.  I can see how someone reading these project log entries might suppose it's a dead project.   It isn't but it also isn't where I wish it were.

The good news is that the code is essentially done and the camera works.  Shown above with a streaming display also shown on the iOS mobile app.  I am currently working toward FW 3.0 for tCam-Mini and will include its new functionality in tCam as well and that will be tCam FW 1.0 running on gCore.

In addition I'm working with Group Gets to bring gCore to market as a general purpose ESP32 development board for building GUI-based battery powered gadgets.  The gCore design is done and I even have an Arduino library and a bunch of demos written for it.  I think it's going to be very cool.

The frustrating news is the parts shortage is making it hard to build gCore.  It may end up with a slight redesign to accommodate available parts. 

I am considering pushing the code to github even though it won't be super useful without gCore (or some modification).

Here are a few more pics showing how tCam is easily built using a Revision 4 tCam-Mini and gCore in a laser-cut enclosure.  The jumper on tCam-Mini configures it to use the hardware serial interface to communicate with gCore.  Hopefully there will also be a 3D printed enclosure design.

Despite the ongoing parts armageddon, the gang at Group Gets managed to find enough parts to build the new Revision 4 tCam-Mini (Rev 3 won't see the light of day outside of 3 prototypes).

They even surprised me by creating a version that supports an external antenna - great for applications like flying the camera on a drone.

Photos courtesy of Group Gets.  The new cameras can be found here and here.

This version is a bit modernized with a USB-C connector and has connections for the new direct hardware interface supported in firmware versions 2.0 and beyond.  Designed so I could use tCam-Mini as the imager for tCam, the hardware interface allows direct connection of tCam-Mini to another micro-controller or SBC.  It allows tCam-Mini to handle all Lepton VoSPI related issues and provides the same high-level interface as WiFi.  It consists of a serial port running at 230,400 baud and a slave SPI port.  The same json commands and responses are transmitted over the serial port with one exception.  Instead of sending an image over the serial port (which would be too slow) the camera sends a new "image_ready" json packet that also includes the length of the image.  Then the controller can use a SPI master to read the image from the slave SPI port on tCam-Mini.  Full details can be found on the github readme.  The 3.3v logic-level hardware interface is activated when the MODE input is detected low at boot (otherwise the WiFi interface is activated).

The github repository has also been updated with the new hardware design files.