Abstract:Countless RGB cameras are ubiquitously distributed in our daily lives, serving to perceive and depict the diverse colors of the world. Reconstructing hyperspectral images (HSI) from these trichromatic cameras emerges as a promising solution to address the limitations of existing, costly hyperspectral imaging systems. The performance of HSI reconstruction relies heavily on the camera spectral response (CSR). Thus, designing a better CSR and putting it into practice is the critical issue for RGB-based HSI reconstruction. However, the CSR curves designed in the existing works are overly random, making them challenging to manufacture directly. Additionally, the designed CSR curves require modifications to the camera hardware, resulting in the loss of RGB imaging functionality. In this paper, we propose a hyperspectral imaging system, which involves enhancing the CSR curve of existing RGB cameras and preserving RGB imaging functionality by adding a learnable physics-based spectral filter. Specifically, we first parameterize the spectral filter transmittance as a function of the filter thicknesses, based on the physical constraints of the multilayer interference principle. Then, we propose a joint optimization framework in which the thicknesses of the filter and the hyperspectral reconstruction network are optimized. In this manner, the thicknesses of the filter are obtained and used to manufacture the filter directly. Finally, we construct a prototype and verify the benefits of our spectral filter design method through experiments including both synthetic data and real images.

Wavelength encoding spectral imaging based on the combination of deeply learned filters and an RGB camera

Deeply Learned Broadband Encoding Stochastic Hyperspectral Imaging.

Spectral Imaging with Deep Learning.

Fabry-Perot Multilayer for Spectral Imaging Applications

Compressive Single-Pixel Hyperspectral Imaging Using RGB Sensors

Hyperspectral Imaging Bioinspired by Chromatic Blur Vision in Color Blind Animals

Color Router-based Long-wave Infrared Multispectral Imaging

Deep‐Learned Broadband Encoding Stochastic Filters for Computational Spectroscopic Instruments

Learning based compressive snapshot spectral light field imaging with RGB sensors

High-resolution true-color imaging based on wavelength-multiplexed far-field spatial-frequency shift imaging

Fast hyperspectral single-pixel imaging via frequency-division multiplexed illumination

Convolutional Sparse Coding for RGB+NIR Imaging.

Spatial-frequency Encoded Imaging of Multangular and Multispectral Images

Spectral-depth imaging with deep learning based reconstruction.

Real-time machine learning-enhanced hyperspectro-polarimetric imaging via an encoding metasurface

Deep-learning-based hyperspectral imaging through a RGB camera

Study on Spectral Encoded Computational Ghost Imaging

Learning a Physics-Based Filter Attachment for Hyperspectral Imaging with RGB Cameras

Lensless Multispectral Camera Based on a Coded Aperture Array

Deep-learning Enables Single-Pixel Spectral Imaging

Ultra-high-speed four-dimensional hyperspectral imaging