Close
0%
0%

Lepton 3.5 Thermal Imaging Camera

Documenting my experiments with the FLIR Lepton 3.5 thermal imaging camera.

Similar projects worth following
I bought a FLIR Lepton 3.5 mounted on the Pure Engineering breakout after using a friend's thermal imaging camera to analyze heat generation on a PCB I designed. The Lepton 3.5 can output absolute temperature data (Radiometric) at 160x120 pixel resolution. I decided a thermal imaging camera would be a good tool to have and decided to build my own. I started with a Teensy 3.2 based test platform, turned to a Beaglebone Black with a short stop in Raspberry Pi land, and finally developed a set of thermal imaging cameras based around the ESP32. I hope the documentation in this project is helpful to others who might also want to play with these amazing devices.

My original long-term goal was to create a capable thermal imaging camera using the Beaglebone Black and a 7" LCD cape as the platform matching some of the features of high end commercial products.  However it soon became obvious that I'd need a simpler platform to learn how to use the Lepton module when I started reading the documentation and playing with the various demo codebases.  This is the story of my thermal imaging camera journey.

I read about a lot of other great projects online to get going.  Pure Engineering and Group Gets are to be commended for making these devices available to makers.  Max Ritter's DIY Thermocam and Damien Walsh's Leptonic are really well done and instructive.  Both Max and Damien were very gracious when I sent them various questions when I was getting going.

I ended up starting with a Teensy 3.2 based prototype with a Lepton breakout board, before moving on the Raspberry Pi, then the exciting world of PRUs in the Beaglebone SBCs before finally settling on the ESP32 for my final design (actually designs).

tcamViewer-release.apk

tCamViewer Android App version 1.0

package-archive - 16.47 MB - 11/25/2023 at 16:11

Download

teensy_schematic.pdf

Test platform schematic

Adobe Portable Document Format - 26.67 kB - 07/10/2018 at 19:11

Preview

View all 18 components

  • Android App finally back in Play Store

    Dan Julio04/20/2024 at 16:52 0 comments

    I won't bore you with the gruesome details but the Android App for tCam-Mini and tCam is back in the Google Play Store here.  The APK is also available at the release repo here if you prefer to side-load.  The app has the features described before, probably the most interesting new feature is the ability to export mp4 video files.  Documentation can be found in the release repo or my website.  I am so sorry it took so long to get this restored.

  • Android App Update

    Dan Julio11/25/2023 at 16:11 0 comments

    As you know the Android App was graciously written by someone else and they have been hard at work updating the app in their spare time.  Unfortunately, at the same time there was an issue with the credit card associated with their google account and the older version of the app disappeared from the App store.

    While they sort that out I got a copy of the version 1.0 APK for those of you comfortable side-loading the app into your phone and have posted it in the download section of this project.

    It has a bunch of new cool features.

    • Various bug fixes
    • Ability to record tmjsn video files and export mp4 videos
    • Ability to playback videos in the browse window
    • Revamped export metadata layout
    • Support for Bonjour camera discovery

  • An alternate firmware for tCam-Mini yields analog video out

    Dan Julio09/27/2023 at 01:09 0 comments

    I recently came across a library on github written by a mysterious user named aquaticus that uses the ESP32 I2S and DAC peripherals to generate a monochrome analog video output.

    It never ceases to amaze me what people are doing with this chip and I immediately thought about mashing it up with the existing tCam-Mini firmware.  I was in part motivated by tear downs of old thermal imaging cameras I have seen in the Thermal Imaging Forum on Dave Jones' website.  Some of these old cameras where incredibly complex (and costly) and now we can generate similar results with the cheap Lepton and super-cheap ESP32.

    A couple of days effort yielded some great results and it has all been pushed up to github if you want to take a look.

    The ESP32 DAC is 8-bits referenced to the 3.3V supply.  Video levels into a 75-ohm load are typically 0 - 1V with blanking at zero, black at around 0.3V and full white at around 1V.

    The video library generates buffers full of data constructed to properly generate the hsync and vsync amongst the video data and then uses the I2S to DMA that through the DAC.  The original video library demo code constructed the buffers so the DAC never outputs voltages greater than 1V which means the value of any data sent to it must be less than 1/3.3 * 255 = 77.  With the need for sync this means only about 55 counts available for video data which is not a lot.  Fortunately you can adjust the parameters and I ended up going with voltages between 0 - 2.36V which gives me 128 intensity levels.  More but still not the 256 intensity levels that are usually used.  Because of that I always enable AGC in the lepton so it outputs data with the best contrast.  The resultant image looks OK.  No radiometric data but we can still pull the spot meter temperature from the telemetry.  The additional voltage range requires an external 100 ohm resistor so that the voltage seen by the 75 ohm input display is scaled back to 0 - 1 V.  The video output can be on either DAC pin (GPIO25 or GPIO26).  Fortunately I am already bringing GPIO26 out to the Slave IF port which makes connecting the video very easy.

    The firmware supports a few features.

    1. Works with Lepton 3.0, 3.1R or 3.5. Radiometric Leptons configured with AGC enabled for best image quality. Gets the full 8.7 FPS possible from the Leptons.
    2. 320x240 pixel image display (linearly interpolated from Lepton 160x120 image).
    3. Optionally displays the 2x2 pixel spotmeter and spotmeter temperature in the center of the image for radiometric Leptons.
    4. Optionally displays min/max temperature location markers.
    5. Supports both white-hot and black-hot palettes.
    6. Supports setting emissivity.
    7. Supports both Imperial (°F) and Metric (°C) temperature readouts.
    8. Stores configurable operating parameters in ESP32 Non-volatile flash.
    9. Firmware may be re-compiled with reduced number of gradations to eliminate need for external 100-ohm resistor.

    The button is used to change various operating parameters.  It can cycle through the two palettes, enable or disable the markers and spot meter, set the emissivity and configure the readout units.

    The firmware can draw text over the image using one of two fonts.  An 8x16 pixel 7-segment numeric font is used for the temperature and a 7x10 pixel alphanumeric font used to display some parameters as they are changed.

    The project is configured to support ESP32 Rev 1 and beyond processors so it can also be run on home-brew tCam-Mini cameras if you already have a Lepton breakout board.  But you have to have a ESP32 WROVER module since it has the necessary external PSRAM.

    All in all a fun little diversion.  Here's a camera aimed at some black body radiator temperature references.

    Full instructions and the code are at the new github repo.

  • tCam Firmware Version 1.0

    Dan Julio09/13/2023 at 16:42 0 comments

    I believe in the aphorism that untested code is buggy code.  In all my testing of the tCam firmware it appears I had never started with freshly booted code in AP mode, used the Wifi scanning feature to pick an existing Wifi network, and entered the password for that.  Or if I had I never noticed that the firmware crashed after setting the new Wifi parameters (perhaps because it so quickly reboots).  Turns out there was a condition where the mDNS server was trying to send packets just as the Wifi was being reconfigured which would crash.  Turns out the fix is easy (just disable, re-enable mDNS around wifi reconfigurations) but the bug was not exposed in any other case where wifi was reconfigured.

    That bug fix provided the perfect opportunity to add a new feature, move to IDF version 4.4.4 and fix exotic another bug I found while looking at code and call the release version 1.0.

    The new feature adds a pair of markers on the main display (and playback display) showing the point of minimum and maximum temperature for radiometric images.

    The Min/Max markers can be turned on or off in Settings (mM Makers).

    The new firmware has been pushed to the repo and my website.  It can be easily downloaded many ways: 

    1. Through the Desktop application and sent to the camera over-the-air
    2. Through the tCam Family Serial Updater (available here)
    3. Through the gCore Serial Programmer (available here)

    I'm really happy with how tCam came out and think version 1.0 marks a good feature complete milestone.

  • Desktop Application version 3.2

    Dan Julio09/11/2023 at 17:08 0 comments

    Version 3.2 of the Desktop application is available on my website.  Existing users can use the "Check for updates" function to download it too.

    It's a fairly minor update with a bug fix, slightly enhanced drawing performance and a couple of new features.

    The main addition is the ability to display markers on the image at the location of the minimum and maximum temperature points.  If enabled the min/max temps can be graphed as well.

    Marker temperatures are now also added to the exported images when enabled. The export image layout was tweaked a little to make it all fit.

    I also fixed a bug where the markers were drawn in the wrong location for some exported image resolutions. 

    Mac Users can export mp4 video

    I use a paid-for library from MonkeyBread Software to extend Xojo functionality.  I have been using it for the mDNS functionality used to find cameras on the network.  I noticed that it had a wrapper for Apple's AVfoundation framework.  AVfoundation can create video files so I added the ability in the Mac version of the program to export mp4 video files from a tmjsn file.  The mp4 export option shows up in the Save Dialog box when exporting a tmjsn file.

    Sorry to the Windows and Linux users.  I really want to figure out an export method for you too.

  • Supporting the new Lepton 3.1R

    Dan Julio08/22/2023 at 21:01 0 comments

    After a very long gestation period, Teledyne Flir is in the process of releasing a new Lepton module called the Lepton 3.1R.  It is functionally equivalent to the Lepton 3.5 (radiometric, 160x120 pixel, 8.7 FPS, 12 µm pixel size, f1.1 focal length and < 50 mK sensitivity) but with a wider field of view (95x119° vs 51x71°).  You should shortly be able to buy one from Group Gets among other places.

    The module works fine with tCam-Mini with firmware > 3.0.  Today I just released a new firmware (version 3.2) which will correctly detect the model number and report back the type in the camera status information.  This means that the information screen in tCam and the Camera Information dialog box in the desktop app will correctly display the "Lepton 3.1" imager type and you can get the info via the python driver as well.

    My website and github have been updated to point to the new firmware.  The automatic download functions in the Desktop application and the tCam Family Serial Updater program will fetch it too.

  • A great set of tCam-Mini enclosures from a user

    Dan Julio07/10/2023 at 01:19 0 comments

    I've always admitted my mechanical design skills are rudimentary at best.  And while I love OpenSCAD as a design tool (it fits my programmer's mind), it certainly is more limited in making complex designs.  A tCam-Mini user created a very nice set of enclosures and shared their STL files (and BOM for additional hardware) in his github repo.  He designed his enclosures to accommodate tCam-Mini with an external antenna but they'll also work for the units with an internal antenna.  I definitely encourage you to check them out.  A big shout out to Harry for creating these.  Some pictures from his repo follow.

  • tCamViewer app in Google Play Store

    Dan Julio04/06/2023 at 13:04 0 comments

    I'm very excited to announce that the beta release of the Android tCamViewer app by Jim Turner is available in the Play Store.  There may still be a few issues here and there so please let Jim know via email on his website or log an issue at the github repository.

  • tCam kit available at Group Gets

    Dan Julio03/02/2023 at 22:30 0 comments

    Group Gets has created a store entry for a tCam kit.  You can find it here.  It includes

    • tCam-mini Rev4 (onboard antenna)
    • gCore (pre-flashed with tCam-firmware)
    • 3D Printed Enclosure

    • Hardware:
      • 4x M2x10 machine screws
      • 4x 4-20 x 1/4 screws
      • 8-Pin Female Header
      • 10-Pin Male Header
      • 2-Pin Jumper
    • Lithium Ion Battery 2000mAh

    You'll have to add a Lepton.  I recommend the Lepton 3.5 instead of the Lepton FS because you're guaranteed accurate radiometric (temperature) data from the 3.5.  But the FS is a little cheaper.

    You can also add a Micro-SD card for local image storage.  

    There's also a link to the video showing how to assemble the camera.  Instructions for use and links to all the supporting software in the github repository.

  • Android App Alpha (finally!!!!)

    Dan Julio02/24/2023 at 23:11 0 comments

    Edit 2/25/2023 - The developer decided getting the app into the Play Store alpha program was too complex and will be providing an APK for download.

    Summary

    An Android app capable of displaying images from the tCam cameras is ready for some alpha testing.   If you're interested please reach out to me with that, either in a message here on hackaday.io, or to my personal email address which you can find at my website or via my github page.  I'll get you on the list.  When the developer is ready he or I will notify you when it's ready to download and include instructions how to do that and how to provide feedback.

    Initial limitations

    The app is pretty full featured however it has a known issue when streaming.  The streaming stutters.  The developer knows about this and plans to rewrite the socket handling code to deal with this.  Other than that please submit feedback about other issues or features.

    TL;DR about the Android App

    People have asked for an Android App since tCam-Mini was first released and I don't blame them.  Lots of Android phones out there.  If you've ready back you might see that I originally wanted to use Xojo to take the iOS app I developed last year and release a version for Android when Xojo finally supported sockets on Android.  Xojo's Android effort is moving at a snail's pace and sadly I don't imagine that I can start that port anytime soon.  Unfortunately I don't have the time - and perhaps the high-level language expertise - to learn native Android app development.

    Last summer a software developer I had been working with on another project, Jim T, kindly offered to write an Android app.  It was his way to relearn Android development and he enjoyed playing with the tCam cameras.  It was much more effort than he originally imagined but to my amazement, he stuck with it in his spare time.  I have helped him test the app and given advice and suggestions and he is finally ready for other people to start testing. 

    So if you have a tCam-Mini or are building a tCam, I hope you'll consider playing with this app.

    Some more pics!

View all 60 project logs

Enjoy this project?

Share

Discussions

Nathan Sinclair wrote 02/01/2023 at 00:06 point

Hello Dan. having limited knowledge about sockets, I have a general query.  What is the reason for using ZeroMQ? From the little research I've done so far the message size for even 1 frame would be way too high... am I missing something? 

  Are you sure? yes | no

Dan Julio wrote 02/24/2023 at 22:26 point

Hi Nathan.  Sorry for the tardy reply - hackaday.io doesn't send me notifications even though I've turned them on.  I'm not sure which camera you are referring regarding ZeroMQ.  I did use it with the linux computers (Pi & Beaglebone) for experimentation because I wanted to interoperate with Damien Walsh's leptonic stuff which used it originally.  For the image data I was sending it seems to be efficient enough.

I use raw sockets for the ESP32-based tCam family of cameras and simply treat them as a full duplex communication channel between the camera and any application connecting to it.  There are special out-of-band start-of-packet and end-of-packet delimiters around the json data packets which make it very easy to always have the two ends synchronized and because it's all riding on top of TCP the communication channel is reliable.

  Are you sure? yes | no

sullivan wrote 08/09/2021 at 18:53 point

Hi Dan. Love the project. I have been playing with a Lepton 3.5 on the breakout board 2 and Raspberry Pi 4B. The thermal images I am getting are in the range of 8300, not the radiometric, T-linear mode I am expecting. Do you get images in the 30000 range where the formula: T(degC) = pixval/100 - 273.15? If you are also in the range of 8300, how are you converting to temperature? Thanks in advance for the advice.

  Are you sure? yes | no

Dan Julio wrote 10/06/2021 at 03:15 point

Hey Sullivan, I'm so sorry for this tardy reply.  For some reason I no longer seem to get notifications when someone posts here.  Are you sure you're switching the camera into radiometric mode with TLinear?  If so then what gain level is your lepton running at?  In High gain mode the resolution is 0.01° which means the data is in the 30000 range (e.g. 25°C -> count of 29815) but in Low gain mode the resolution is 0.1°C (25°C -> 2981).  In my cameras I set Radiometry, Radiometry TLinear, AGC Calcs all true.

  Are you sure? yes | no

leoshmu wrote 01/12/2021 at 03:23 point

Thank you for this excellent project. I am curious about pushing the limits of frame rate. Are you able to achieve a full 9Hz frame rate on the thermal video stream, or do you find that the frame rate is limited to less than 9Hz in practice. Could you imagine getting 2 leptons integrated together and offsetting the thermal stream so that they are off by 55ms, allowing for a combined 18hz signal? Perhaps something with a master clock circuit and then controlling each lepton separately based on the clock timing? Also I'm curious, why do you prefer the pure thermal board to the lepton v2 breakout board? 

  Are you sure? yes | no

Dan Julio wrote 02/04/2021 at 03:26 point

Hey Leo, thanks for the kind words.  I have been able to achieve the full 9Hz frame rate using the Beaglebone's PRU (in AGC mode) and with the ESP32 (both radiometric and AGC modes).  With the ESP32 I more-or-less dedicate one of the two CPUs to getting data out of the Lepton (and I use the VSYNC signal).  I will shortly be posting the code for tCam-Mini which is capable of reading the full frame rate (but not quite capable of sending that frame-rate over the WiFi).

Regarding your interesting idea of using two devices:  I'm not sure how to synchronize them.  I think there is a lot of internal processing that ends up affecting when the device is capable of sending a video packet - and things like a FFC also impact its ability to output a valid frame.  I suspect - but have no proof - that it is capable of a higher frame rate but held back in firmware (the sensor response time seems to be around 50 mSec).  I'm not up to it but I've wondered if someone could hack it.

I like the pure thermal board because it's smaller - I don't think there are any performance related differences.  Now, with tCam-Mini, I'm moving to directly mounting the sensor on my own boards.  

  Are you sure? yes | no

Ailsamly wrote 08/18/2020 at 10:30 point

Hi Dan,

I also plan to build a small thermal camera with a  suitable thermal sensor module, I want to connect it to a mobile platform and do building inspection (like water damage and air leakage and other thermal anomalies) .  The thermal sensor will be 5-10cm in front of the wall. I want to follow Max Ritter's project DIY-Thermocam. But I am not sure whether the resolution of Flir lepton can fulfull the requirements of mine. Do you have some ideas? Thank you!

Best,

Leyuan

  Are you sure? yes | no

Dan Julio wrote 12/26/2020 at 19:11 point

Hello Leyuan, I apologize for this tardy reply.  For some reason I never saw your post.  I think the Lepton should work from a distance of 5-10 cm.  However I can't answer your question about the resolution as I'm not sure what the characteristics of the things you are looking for are.  I can only suggest seeing if you could borrow a camera from someone and do some experimentation before committing to your own build.  Good luck though!  Sounds like a cool project.

  Are you sure? yes | no

Richard wrote 05/30/2020 at 16:15 point

Hi Dan,

According to Flir's datasheets, the Lepton uses 3.0V for IO functions, but the Teensy's on board Vreg runs at 3.3V. Did you do any level translation on the VOSPI and I2C interfaces or modify the Teensy to run at 3.0V, or just put 3.3V on the buses? I'm about to start a Lepton project of my own and want to be sure I don't kill my expensive sensor!

Thanks!

  Are you sure? yes | no

Dan Julio wrote 05/30/2020 at 16:49 point

Hey Richard,

You're right, they specify a 2.8-3.1 volt VDDIO.  But they also specify that the maximum on an IO pin is the lesser of VDDIO + 0.6 volts or 4.8 volts (table 18 - Absolute Maximum Ratings).  The Pure Thermal breakout boards have a 2.8 volt regulator feeding VDDIO so the maximum input voltage should be 3.4 volts.

To be honest I didn't pay attention to this spec when I first started.  I saw that the Pure Thermal boards were used with many different 3.3V SBCs so I jumped right in.  It wasn't until later I saw the specs you are referring to.

Although it doesn't seem like good engineering practice, I have run several Lepton 3.5 modules for long periods of time and this does not seem to change their functionality.  One could design their own breakout board for the Lepton and provide 3.1 volts to VDDIO which would lessen the over-voltage condition even more or provide resistor dividers on the SCLK and CSN signals (and pull SDA/SCL to the VDDIO rail).  As I work on what I hope is my final thermal imaging camera I am contemplating a full custom PCB (eliminating the breakout boards) and this is probably what I'll do, especially after your timely email :-)

  Are you sure? yes | no

Avi Cohen wrote 11/14/2019 at 11:45 point

Hi Dan, 

Is the code open to download?

  Are you sure? yes | no

Dan Julio wrote 11/14/2019 at 17:09 point

Hey Avi, 

yes, it's in the github repository.

https://github.com/danjulio/lepton

  Are you sure? yes | no

Sophi Kravitz wrote 07/10/2018 at 17:21 point

hi Dan! This is a cool project! This is happening pretty soon: https://hackaday.io/contest/159585-battery-powered-radiometric-lepton-contest

  Are you sure? yes | no

Dan Julio wrote 07/10/2018 at 18:09 point

Hi Sophi!  I saw that.  I'm not sure I'll have anything for the Lepton 2.5 but I will contribute a port of FLIR's LeptonSDKEmb32OEM library for the Teensy which might be applicable to the ESP32.

  Are you sure? yes | no

Similar Projects

Does this project spark your interest?

Become a member to follow this project and never miss any updates