raspberry pi multi-spectral imaging system

I had been planning this project for a long time. Once I had some free time, I went to TAMI, our local makerspace, and built a high resolution active multispectral imaging system.

The idea is simple. A monochrome camera photographs the same object many times while a different wavelength of light is switched on for every image. Each photograph becomes one spectral band. Put all of the bands together and you can look for details that are difficult or impossible to see in a normal color photograph.

I originally built it to collect image sets of crops, pests, fungus and plant stress.

The light source contains more than 30 different LED wavelengths from approximately 370 nm to 880 nm, plus white LEDs around 2700 K, 6000 K and 10000 K. I arranged each color as a series of four high intensity LEDs on an aluminium plate that also works as a heatsink.

Each LED string is switched by a MOSFET. Three PCA9685 boards at I2C addresses 0x40, 0x41 and 0x42 provide the PWM control. The LEDs are powered through a Mean Well LDD 700H constant current driver.

I also considered replacing the PCA9685 boards with PCA9635 boards for their higher PWM frequency.

The top third of the imaging box was lined with aluminium foil to diffuse the light. The lower two thirds used thin black cardboard to reduce background reflections. Styrofoam could also work as a diffuser and black velvet would probably absorb more stray light, but these were the materials I could find locally.

I tested every LED with a spectrometer before installing it. I also compared the illumination using white A4 paper and a reflectance standard.

The original plan was to add a photodiode and a transimpedance amplifier so the intensity of every LED could be measured and calibrated in a closed loop. I found several possible photodiodes in my parts box and considered the LTC1050 for converting the photodiode current into a voltage.

Camera and enclosure

I drilled a large hole through the center of the aluminium plate for an IDS monochrome camera. I built an adjustable camera mount from pieces of anodized aluminium that I found at TAMI.

Only reflected light from the sample should reach the lens. To block direct light from the LEDs, I fitted a black painted PVC tube between the camera and the sample. The first tube was too long, so the camera could see the end of it and the image looked like it was being viewed through a bore. An adjustable tube would make it easier to match different lenses and fields of view.

I later added a touchscreen and powered everything from the same source. The LED driver used 15 V, the screen used 12 V and the Raspberry Pi used 5 V. A magnetic latch kept the door closed.

The touchscreen used an eGalax controller. My notes for getting it working were to install xserver-xorg-input-evdev, replace libinput with evdev in /etc/X11/xorg.conf, install xinput-calibrator and swap the X and Y axes if required.

The videos show avocado, grapevine and potato leaves. Some bands are strongly absorbed by the leaves, and the early tests were made before the LED intensity and camera exposure were calibrated. I also tried OpenCV CLAHE contrast equalization and displayed the corresponding histograms.

The first Raspberry Pi occasionally lost frames because it did not have enough processing power. I planned to replace it with an NVIDIA Jetson Nano, rebuild the bottom of the enclosure with better light absorbing material, try several colored lasers and fiber bundles for illumination from different angles, and make the complete station smaller and battery powered.

The original Python program steps through the LED channels, switches on one wavelength, captures an image from the IDS camera and then switches that wavelength off. The values called lux in the early code are temporary PCA9685 PWM counts from 0 to 4095, not calibrated lux measurements.

TAMI:

https://telavivmakers.org/index.php/Main_Page

PCA9685 datasheet:

https://cdn-shop.adafruit.com/datasheets/PCA9685.pdf

Mean Well LDD 700H datasheet:

https://www.mouser.com/ds/2/260/LDD-H-spec-766227.pdf

CLAHE:

https://docs.opencv.org/3.1.0/d5/daf/tutorial_py_histogram_equalization.html

Hackaday later published the project here:

https://hackaday.com/2022/08/25/multispectral-imaging-system-built-with-raspberry-pi/#comments

NASA MIDAR:

https://www.nasa.gov/ames/las/midar

Videos:

https://www.youtube.com/watch?v=FnvTUBJZrsQ

https://www.youtube.com/watch?v=aXgGQMTvtUQ

Build photos and the original capture code below.

Original capture code, cleaned up

from __future__ import division

import ctypes
import datetime

import Adafruit_PCA9685
from pyueye import ueye


pwm_boards = [
    Adafruit_PCA9685.PCA9685(address=0x40),
    Adafruit_PCA9685.PCA9685(address=0x41),
    Adafruit_PCA9685.PCA9685(address=0x42),
]

for board in pwm_boards:
    board.set_pwm_freq(1000)

specimen = "avo_leaf"
led_channels = list(range(32))

# Temporary PCA9685 PWM counts from 0 to 4095. These are not lux values.
pwm_counts = list(range(1900, 1932))


def set_led(global_channel, value):
    board_number = global_channel // 16
    local_channel = global_channel % 16
    pwm_boards[board_number].set_pwm(local_channel, 0, value)


for channel, pwm_value in zip(led_channels, pwm_counts):
    set_led(channel, pwm_value)

    hcam = ueye.HIDS(1)
    image_memory = ueye.c_mem_p()
    memory_id = ueye.c_int()
    window = ctypes.c_void_p()

    ueye.is_InitCamera(hcam, window)

    sensor_info = ueye.SENSORINFO()
    ueye.is_GetSensorInfo(hcam, sensor_info)
    ueye.is_AllocImageMem(
        hcam,
        sensor_info.nMaxWidth,
        sensor_info.nMaxHeight,
        24,
        image_memory,
        memory_id,
    )
    ueye.is_SetImageMem(hcam, image_memory, memory_id)

    result = ueye.is_FreezeVideo(hcam, ueye.IS_WAIT)
    if result == 0:
        print("camera")
    else:
        print("____problem!!!____")

    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S_%f")
    filename = "{}_led_{}_cam_{}.bmp".format(
        specimen, channel + 1, timestamp
    )

    file_params = ueye.IMAGE_FILE_PARAMS()
    file_params.pwchFileName = filename
    file_params.nFileType = ueye.IS_IMG_BMP
    file_params.ppcImageMem = None
    file_params.pnImageID = None

    result = ueye.is_ImageFile(
        hcam,
        ueye.IS_IMAGE_FILE_CMD_SAVE,
        file_params,
        ueye.sizeof(file_params),
    )

    if result == 0:
        print("saved", filename)
    else:
        print("____problem!!!____")

    ueye.is_FreeImageMem(hcam, image_memory, memory_id)
    ueye.is_ExitCamera(hcam)
    set_led(channel, 0)

Original parts and reference links

• pca9635
• FDS010
• LTC1050
• raspberry pi
• nvidia jetson nano

Photos and videos