Abstract:Compressive spectral imaging (CSI) has attracted significant attention since it employs synthetic apertures to codify spatial and spectral information, sensing only 2D projections of the 3D spectral image. However, these optical architectures suffer from a trade-off between the spatial and spectral resolution of the reconstructed image due to technology limitations. To overcome this issue, compressive spectral image fusion (CSIF) employs the projected measurements of two CSI architectures with different resolutions to estimate a high-spatial high-spectral resolution. This work presents the fusion of the compressive measurements of a low-spatial high-spectral resolution coded aperture snapshot spectral imager (CASSI) architecture and a high-spatial low-spectral resolution multispectral color filter array (MCFA) system. Unlike previous CSIF works, this paper proposes joint optimization of the sensing architectures and a reconstruction network in an end-to-end (E2E) manner. The trainable optical parameters are the coded aperture (CA) in the CASSI and the colored coded aperture in the MCFA system, employing a sigmoid activation function and regularization function to encourage binary values on the trainable variables for an implementation purpose. Additionally, an unrolling-based network inspired by the alternating direction method of multipliers (ADMM) optimization is formulated to address the reconstruction step and the acquisition systems design jointly. Finally, a spatial-spectral inspired loss function is employed at the end of each unrolling layer to increase the convergence of the unrolling network. The proposed method outperforms previous CSIF methods, and experimental results validate the method with real measurements.

Residual Degradation Learning Unfolding Framework with Mixing Priors across Spectral and Spatial for Compressive Spectral Imaging

Spectral Imaging with Deep Learning.

Deep-learning-based CASSI Reconstruction with Optical Aberration Correction

Degradation Estimation Recurrent Neural Network With Local and Non-Local Priors for Compressive Spectral Imaging

Snapshot Spectral Imaging Based on Aberration Model-Driven Deep Learning

Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging

Simultaneous Coded Aperture and Dictionary Optimization in Compressive Spectral Imaging Via Coherence Minimization.

Convolutional Neural Network Based Computational Imaging Spectroscopy

Hyperspectral image recovery based on fusion of coded aperture snapshot spectral imaging and RGB images by guided filtering

Residual image recovery method based on the dual-camera design of a compressive hyperspectral imaging system

Hybrid Compressive Spectral Imaging with Coded Aperture Optimization Based on Coherence Minimization

Proximal Gradient Descent Unfolding Dense-spatial Spectral-attention Transformer for Compressive Spectral Imaging

Hyperspectral Image Reconstruction of SD-CASSI Based on Nonlocal Low-Rank Tensor Prior

Latent Diffusion Prior Enhanced Deep Unfolding for Snapshot Spectral Compressive Imaging

D$^\text{2}$UF: Deep Coded Aperture Design and Unrolling Algorithm for Compressive Spectral Image Fusion

Hyperspectral Compressive Snapshot Reconstruction via Coupled Low-Rank Subspace Representation and Self-Supervised Deep Network

Deep-learning Enables Single-Pixel Spectral Imaging

Coarse-Fine Spectral-Aware Deformable Convolution For Hyperspectral Image Reconstruction

Efficient One-Step Diffusion Refinement for Snapshot Compressive Imaging

RCUMP: Residual Completion Unrolling With Mixed Priors for Snapshot Compressive Imaging