Abstract:Coded Aperture Snapshot Spectral Imaging (CASSI) reconstruction aims to recover the 3D spatial-spectral signal from 2D measurement. Existing methods for reconstructing Hyperspectral Image (HSI) typically involve learning mappings from a 2D compressed image to a predetermined set of discrete spectral bands. However, this approach overlooks the inherent continuity of the spectral information. In this study, we propose an innovative method called Spectral-wise Implicit Neural Representation (SINR) as a pioneering step toward addressing this limitation. SINR introduces a continuous spectral amplification process for HSI reconstruction, enabling spectral super-resolution with customizable magnification factors. To achieve this, we leverage the concept of implicit neural representation. Specifically, our approach introduces a spectral-wise attention mechanism that treats individual channels as distinct tokens, thereby capturing global spectral dependencies. Additionally, our approach incorporates two components, namely a Fourier coordinate encoder and a spectral scale factor module. The Fourier coordinate encoder enhances the SINR's ability to emphasize high-frequency components, while the spectral scale factor module guides the SINR to adapt to the variable number of spectral channels. Notably, the SINR framework enhances the flexibility of CASSI reconstruction by accommodating an unlimited number of spectral bands in the desired output. Extensive experiments demonstrate that our SINR outperforms baseline methods. By enabling continuous reconstruction within the CASSI framework, we take the initial stride toward integrating implicit neural representation into the field.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to achieve continuous spectral information recovery in hyperspectral image (HSI) reconstruction in the Coded Aperture Snapshot Spectral Imaging (CASSI) system, rather than the traditional discrete spectral band reconstruction method. Existing HSI reconstruction methods usually involve mapping from 2D compressed images to preset discrete spectral bands. This method ignores the inherent continuity of spectral information, resulting in the loss of spectral features and a decline in reconstruction accuracy. In addition, the model needs to be retrained for different numbers of spectral bands, which greatly limits the practical application of the CASSI system. To solve these problems, the author proposes an innovative method - Spectral - wise Implicit Neural Representation (SINR), aiming to achieve HSI reconstruction by introducing a continuous spectral magnification process, thus supporting custom magnification factors and being able to handle any number of spectral bands. SINR enhances the ability to learn high - frequency information and adapt to different numbers of spectral bands through the concept of implicit neural representation, especially by introducing the spectral attention mechanism, Fourier coordinate encoder, and spectral scale factor module. These designs enable SINR to flexibly reconstruct HSIs with different spectral resolutions without retraining, significantly improving the flexibility and practicality of the CASSI system. Specifically, the main contributions of SINR include: 1. For the first time, implicit neural representation is introduced into HSI reconstruction in the CASSI system, achieving continuous spectral reconstruction, and only one model needs to be trained to reconstruct any number of spectral bands. 2. A new spectral attention mechanism is proposed to capture long - distance relationships along the spectral dimension, further improving the reconstruction quality. 3. Structural designs such as the Fourier coordinate encoder and scale factor module are introduced. These components improve the adaptability of implicit functions to different scales and frequency information. Through these innovations, SINR not only significantly outperforms the baseline methods in comprehensive quantitative indicators and perceptual quality but also is superior to existing methods in spectral fidelity, even when compared with models trained for a specific number of spectral bands. This marks a significant progress in achieving HSI reconstruction in the CASSI system.

Spectral-wise Implicit Neural Representation for Hyperspectral Image Reconstruction

Deep-learning-based CASSI Reconstruction with Optical Aberration Correction

Snapshot Spectral Imaging Based on Aberration Model-Driven Deep Learning

Hyperspectral image recovery based on fusion of coded aperture snapshot spectral imaging and RGB images by guided filtering

Hyperspectral Compressive Snapshot Reconstruction via Coupled Low-Rank Subspace Representation and Self-Supervised Deep Network

Implicit Neural Representation Learning for Hyperspectral Image Super-Resolution

Hybrid Compressive Spectral Imaging with Coded Aperture Optimization Based on Coherence Minimization

Convolutional Neural Network Based Computational Imaging Spectroscopy

Continuous Spatial-Spectral Reconstruction Via Implicit Neural Representation

Hyperspectral Image Reconstruction from RGB Input Through Highlighting Intrinsic Properties

Coarse-Fine Spectral-Aware Deformable Convolution For Hyperspectral Image Reconstruction

HyperReconNet: Joint Coded Aperture Optimization and Image Reconstruction for Compressive Hyperspectral Imaging.

Reconstructing hyperspectral images from rgb inputs based on intrinsic image decomposition

Hyperspectral Image Reconstruction of SD-CASSI Based on Nonlocal Low-Rank Tensor Prior

DS PN: A Two-Stage Direction-Aware Spectral-Spatial Perceptual Network for Hyperspectral Image Reconstruction

DS $^{2}$ PN: A Two-Stage Direction-Aware Spectral-Spatial Perceptual Network for Hyperspectral Image Reconstruction

Hyperspectral Image Super-Resolution with 1D-2D Attentional Convolutional Neural Network.

Fast Hyperspectral Image Recovery via Non-iterative Fusion of Dual-Camera Compressive Hyperspectral Imaging

Hyperspectral Image Reconstruction Via Patch Attention Driven Network

Progressive Content-aware Coded Hyperspectral Compressive Imaging

Progressive Content-Aware Coded Hyperspectral Snapshot Compressive Imaging