Lepton 3.5 Thermal Imaging Camera

I won't bore you with the gruesome details but the Android App for tCam-Mini and tCam is back in the Google Play Store here.  The APK is also available at the release repo here if you prefer to side-load.  The app has the features described before, probably the most interesting new feature is the ability to export mp4 video files.  Documentation can be found in the release repo or my website.  I am so sorry it took so long to get this restored.

As you know the Android App was graciously written by someone else and they have been hard at work updating the app in their spare time.  Unfortunately, at the same time there was an issue with the credit card associated with their google account and the older version of the app disappeared from the App store.

While they sort that out I got a copy of the version 1.0 APK for those of you comfortable side-loading the app into your phone and have posted it in the download section of this project.

It has a bunch of new cool features.

• Various bug fixes
• Ability to record tmjsn video files and export mp4 videos
• Ability to playback videos in the browse window
• Revamped export metadata layout
• Support for Bonjour camera discovery

I recently came across a library on github written by a mysterious user named aquaticus that uses the ESP32 I2S and DAC peripherals to generate a monochrome analog video output.

It never ceases to amaze me what people are doing with this chip and I immediately thought about mashing it up with the existing tCam-Mini firmware.  I was in part motivated by tear downs of old thermal imaging cameras I have seen in the Thermal Imaging Forum on Dave Jones' website.  Some of these old cameras where incredibly complex (and costly) and now we can generate similar results with the cheap Lepton and super-cheap ESP32.

A couple of days effort yielded some great results and it has all been pushed up to github if you want to take a look.

The ESP32 DAC is 8-bits referenced to the 3.3V supply.  Video levels into a 75-ohm load are typically 0 - 1V with blanking at zero, black at around 0.3V and full white at around 1V.

The video library generates buffers full of data constructed to properly generate the hsync and vsync amongst the video data and then uses the I2S to DMA that through the DAC.  The original video library demo code constructed the buffers so the DAC never outputs voltages greater than 1V which means the value of any data sent to it must be less than 1/3.3 * 255 = 77.  With the need for sync this means only about 55 counts available for video data which is not a lot.  Fortunately you can adjust the parameters and I ended up going with voltages between 0 - 2.36V which gives me 128 intensity levels.  More but still not the 256 intensity levels that are usually used.  Because of that I always enable AGC in the lepton so it outputs data with the best contrast.  The resultant image looks OK.  No radiometric data but we can still pull the spot meter temperature from the telemetry.  The additional voltage range requires an external 100 ohm resistor so that the voltage seen by the 75 ohm input display is scaled back to 0 - 1 V.  The video output can be on either DAC pin (GPIO25 or GPIO26).  Fortunately I am already bringing GPIO26 out to the Slave IF port which makes connecting the video very easy.

The firmware supports a few features.

1. Works with Lepton 3.0, 3.1R or 3.5. Radiometric Leptons configured with AGC enabled for best image quality. Gets the full 8.7 FPS possible from the Leptons.
2. 320x240 pixel image display (linearly interpolated from Lepton 160x120 image).
3. Optionally displays the 2x2 pixel spotmeter and spotmeter temperature in the center of the image for radiometric Leptons.
4. Optionally displays min/max temperature location markers.
5. Supports both white-hot and black-hot palettes.
6. Supports setting emissivity.
7. Supports both Imperial (°F) and Metric (°C) temperature readouts.
8. Stores configurable operating parameters in ESP32 Non-volatile flash.
9. Firmware may be re-compiled with reduced number of gradations to eliminate need for external 100-ohm resistor.

The button is used to change various operating parameters.  It can cycle through the two palettes, enable or disable the markers and spot meter, set the emissivity and configure the readout units.

The firmware can draw text over the image using one of two fonts.  An 8x16 pixel 7-segment numeric font is used for the temperature and a 7x10 pixel alphanumeric font used to display some parameters as they are changed.

The project is configured to support ESP32 Rev 1 and beyond processors so it can also be run on home-brew tCam-Mini cameras if you already have a Lepton breakout board.  But you have to have a ESP32 WROVER module since it has the necessary external PSRAM.

All in all a fun little diversion.  Here's a camera aimed at some black body radiator temperature references.

Full instructions and the code are at the new github repo.

I believe in the aphorism that untested code is buggy code.  In all my testing of the tCam firmware it appears I had never started with freshly booted code in AP mode, used the Wifi scanning feature to pick an existing Wifi network, and entered the password for that.  Or if I had I never noticed that the firmware crashed after setting the new Wifi parameters (perhaps because it so quickly reboots).  Turns out there was a condition where the mDNS server was trying to send packets just as the Wifi was being reconfigured which would crash.  Turns out the fix is easy (just disable, re-enable mDNS around wifi reconfigurations) but the bug was not exposed in any other case where wifi was reconfigured.

That bug fix provided the perfect opportunity to add a new feature, move to IDF version 4.4.4 and fix exotic another bug I found while looking at code and call the release version 1.0.

The new feature adds a pair of markers on the main display (and playback display) showing the point of minimum and maximum temperature for radiometric images.

The Min/Max markers can be turned on or off in Settings (mM Makers).

The new firmware has been pushed to the repo and my website.  It can be easily downloaded many ways: 

1. Through the Desktop application and sent to the camera over-the-air
2. Through the tCam Family Serial Updater (available here)
3. Through the gCore Serial Programmer (available here)

I'm really happy with how tCam came out and think version 1.0 marks a good feature complete milestone.

Version 3.2 of the Desktop application is available on my website.  Existing users can use the "Check for updates" function to download it too.

It's a fairly minor update with a bug fix, slightly enhanced drawing performance and a couple of new features.

The main addition is the ability to display markers on the image at the location of the minimum and maximum temperature points.  If enabled the min/max temps can be graphed as well.

Marker temperatures are now also added to the exported images when enabled. The export image layout was tweaked a little to make it all fit.

I also fixed a bug where the markers were drawn in the wrong location for some exported image resolutions. 

Mac Users can export mp4 video

I use a paid-for library from MonkeyBread Software to extend Xojo functionality.  I have been using it for the mDNS functionality used to find cameras on the network.  I noticed that it had a wrapper for Apple's AVfoundation framework.  AVfoundation can create video files so I added the ability in the Mac version of the program to export mp4 video files from a tmjsn file.  The mp4 export option shows up in the Save Dialog box when exporting a tmjsn file.

Sorry to the Windows and Linux users.  I really want to figure out an export method for you too.

After a very long gestation period, Teledyne Flir is in the process of releasing a new Lepton module called the Lepton 3.1R.  It is functionally equivalent to the Lepton 3.5 (radiometric, 160x120 pixel, 8.7 FPS, 12 µm pixel size, f1.1 focal length and < 50 mK sensitivity) but with a wider field of view (95x119° vs 51x71°).  You should shortly be able to buy one from Group Gets among other places.

The module works fine with tCam-Mini with firmware > 3.0.  Today I just released a new firmware (version 3.2) which will correctly detect the model number and report back the type in the camera status information.  This means that the information screen in tCam and the Camera Information dialog box in the desktop app will correctly display the "Lepton 3.1" imager type and you can get the info via the python driver as well.

My website and github have been updated to point to the new firmware.  The automatic download functions in the Desktop application and the tCam Family Serial Updater program will fetch it too.

I've always admitted my mechanical design skills are rudimentary at best.  And while I love OpenSCAD as a design tool (it fits my programmer's mind), it certainly is more limited in making complex designs.  A tCam-Mini user created a very nice set of enclosures and shared their STL files (and BOM for additional hardware) in his github repo.  He designed his enclosures to accommodate tCam-Mini with an external antenna but they'll also work for the units with an internal antenna.  I definitely encourage you to check them out.  A big shout out to Harry for creating these.  Some pictures from his repo follow.

I'm very excited to announce that the beta release of the Android tCamViewer app by Jim Turner is available in the Play Store.  There may still be a few issues here and there so please let Jim know via email on his website or log an issue at the github repository.

Group Gets has created a store entry for a tCam kit.  You can find it here.  It includes

• tCam-mini Rev4 (onboard antenna)
• gCore (pre-flashed with tCam-firmware)

• 3D Printed Enclosure

• Hardware:
  • 4x M2x10 machine screws
  • 4x 4-20 x 1/4 screws
  • 8-Pin Female Header
  • 10-Pin Male Header
  • 2-Pin Jumper
• Lithium Ion Battery 2000mAh

You'll have to add a Lepton.  I recommend the Lepton 3.5 instead of the Lepton FS because you're guaranteed accurate radiometric (temperature) data from the 3.5.  But the FS is a little cheaper.

You can also add a Micro-SD card for local image storage.  

There's also a link to the video showing how to assemble the camera.  Instructions for use and links to all the supporting software in the github repository.

Edit 2/25/2023 - The developer decided getting the app into the Play Store alpha program was too complex and will be providing an APK for download.

Summary

An Android app capable of displaying images from the tCam cameras is ready for some alpha testing.   If you're interested please reach out to me with that, either in a message here on hackaday.io, or to my personal email address which you can find at my website or via my github page.  I'll get you on the list.  When the developer is ready he or I will notify you when it's ready to download and include instructions how to do that and how to provide feedback.

Initial limitations

The app is pretty full featured however it has a known issue when streaming.  The streaming stutters.  The developer knows about this and plans to rewrite the socket handling code to deal with this.  Other than that please submit feedback about other issues or features.

TL;DR about the Android App

People have asked for an Android App since tCam-Mini was first released and I don't blame them.  Lots of Android phones out there.  If you've ready back you might see that I originally wanted to use Xojo to take the iOS app I developed last year and release a version for Android when Xojo finally supported sockets on Android.  Xojo's Android effort is moving at a snail's pace and sadly I don't imagine that I can start that port anytime soon.  Unfortunately I don't have the time - and perhaps the high-level language expertise - to learn native Android app development.

Last summer a software developer I had been working with on another project, Jim T, kindly offered to write an Android app.  It was his way to relearn Android development and he enjoyed playing with the tCam cameras.  It was much more effort than he originally imagined but to my amazement, he stuck with it in his spare time.  I have helped him test the app and given advice and suggestions and he is finally ready for other people to start testing. 

So if you have a tCam-Mini or are building a tCam, I hope you'll consider playing with this app.

Some more pics!