Multispectral imaging smartphone camera

For multispectral image analysis, I am using the set of wheels with bandpass filters from 365--940nm. Taking a full set of photos allows software to map the spectral curve of various materials in that range, and, depending on the material, can reasonably identify or narrow down ID possibilities.

Limited a bit by space, these are some sample images taken with the visible range filter wheel:

450nm:

650nm:

After taking a full set of images with each bandpass filter, I will put them through multispectral imaging software to process the spectral curves and allow me to narrow down material IDs.

Got to test out the LED modules. I made a UV (365nm) and an IR (850nm) module, both of which serve different purposes and enable different types of analysis.

The IR and UV light modules snap onto the magnetic back and plug into the USB port.

UV fluorescence, showing optical brighteners in the paper.

It's Christmas already!

I got the full set of bandpass filters in the mail, and got to start assembling specific filter wheels.

The filters range from 365nm-940nm, roughly every 50nm. I started with the IR wheel, which includes 940, 900, 850, and 780nm. This allows me to separate out materials based on their opacities at these wavelengths.

Assembling filters wheels with all of the bandpass filters will allow me to use the smartphone to do multiband imaging for pigment/material analysis.

I made a PCB for prototyping the LED modules. This is my first PCB, so it's very simple, but the through hole points allow for soldering in header pins & testing various configurations of resistors/potentiometers/switches, etc.

I used a few USB-C OTG cables to wire up to the PCB, cutting out the data wires.

First, the IR module: I made the 850nm module first, since that is the wavelength of bandpass filter that has been the most consistently useful in my tests. The PCB has an array of 9 LEDs in a grid, which has given enough output for photos up to 5-6 feet away from the surface.

Next, the UV module, which needed to be mounted on top, without the diffuser over it, because the shorter UV wavelengths won't go through the diffuser.

My end goal is to integrate with the Photopea API for automatic photo processing for various types of technical imaging.

So far, I have made a set of actions with can be loaded into Photopea, and opened whenever the device opens the web page. Add a couple of other steps, and this allows for creation of a FCIR photo in as little as 2 minutes.

First: take and load in the visible light photo and IR photo overlaid on top.

Second: apply actions which auto align and swap channels.

Third: copy and past IR layer into Red channel of vis layer. Done!

False color infrared photo.

I hope to work out how to apply these actions via the Photopea API, which will make FCIR even easier and quicker.

I got the opportunity to test the camera at the MET, and got a few good examples of Infrared Reflectography and inpainting on a few paintings.

.                                                             Visible .	Infrared Reflectography (notice the dark line through the middle--that's inpainting at a split in the wood panel)
Visible	Infrared Reflectography                (notice pencil grid lines)
.                                                            Visible .	Infrared Reflectography (notice faint vertical pencil lines around most of the figures)

I've been working a lot on the case, trying to find ways to slim it down and increase its functionality. I ended up designing a magnetic mount that allows attachment of various modules, and only adds 4mm to the overall thickness. Overall, it's no bigger than a standard battery case.

I used particularly strong magnets to give a secure hold, and the raised design helps to align the mount and hold everything in place when attached. The only way to pull it apart is straight out; the effects of side knocks & twisting are minimized.

The case includes the filter wheel, a tripod mount, and a self-aligning magnetic mount. I'm planning on making 'magmount' tiles for additional modules, my Openflexure microscope, and a photogrammetry turntable.

I've been working on modeling a case to include the filter wheel, removable LED light panels, and a tripod mount. It's been a slow process because I've had to learn Fusion 360 for this project.

But I finally got the first test prints, and it's all fitting together better than I had expected for a first go around.

                                                 Mounted on a tripod, with LED module. 

                                   I only have white LEDs wired up, but next is UV and IR LEDs.

Removal of the IR filter took a lot of trial and error--some camera modules are less suitable for modification than others, and I had to figure out a good process by which to disassemble them.

For the Pixel 3a camera, I tried using a scalpel to separate the sensor from the lens housing, but this ended up breaking some very tiny solder joins that connected to the autofocus mechanism in the lens housing. If it didn't have autofocus, it would have been no problem!

I tried soldering those broken joins, but they were too small and beyond my soldering skills.

In order to avoid breaking them in the first place, I settled on using a heated 3d printer bed to soften the adhesive around the lens (15 min. on 60C), and using a small pair of pliers to twist and remove the lens. This gave me access to the IR filter, which is glued into a square frame just above the sensor.

To remove the IR filter, I glued a small section of ABS 3d printer filament to it to pull it out.

It ended up breaking instead.

But, after carefully removing the rest of the broken IR filter, I was able to twist the lens back in place and put the camera module back into the phone without an issue.

                                               Disassembled phone and camera modules

                                    Camera module with lens removed, showing broken IR filter

                                                  Camera module after removal of IR filter.

I'm going to try printing a small tool to adhere better to the IR filter, and hopefully with some heat, I'll be able to pull out the filter intact.

I have managed to safely remove the integrated IR filter in the Google Pixel 3a phone--the method: a heated 3d printer bed, set at 60C, is used to soften the adhesive that holds the lens in place. After about 15-20 minutes, the lens is carefully and slowly twisted with a small set of pliers until it comes out.

To remove the filter, I have been using a small amount of adhesive (superglue or a quick-setting epoxy) on the end of a small section of 3d printer filament. I glue this piece onto the filter itself, allow it to cure, then set the camera back on the printer bed and pull the filter out.

I have broken several filters, though, and have had to carefully remove all traces of broken glass from inside the camera module.

But despite the fact that I need to refine my technique, it works!

The filter wheel is printed in PETG, and designed to house 4 filters that enable up to 7 types of technical imaging: Visible, Raking light, UV fluorescence, UV reflectography, IR reflectography, IR fluorescence, and Polarized.

                                                           Painting in visible light

                                           Same painting in UV reflectography. Note the difference                                                                                     between 2 different white paints.