Abstract:Spectrometers can obtain fine spectral reflectance of substances through high-resolution spectral sampling, which has important advantages in identifying the categories of ground substances. As a complement to high-cost spectrometers, reconstruction from RGB images provides an economical and convenient way to acquire hyperspectral images. Although the current mainstream reconstruction methods based on deep convolution network can directly learn the inverse reconstruction mapping in data-driven way, they still have shortcomings in the representation of multi-scale and long-range spatial-spectral correlations within hyperspectral images. To conquer these issues, we propose a novel multi-scale spatial-spectral Transformer network for spectral reconstruction. The proposed network consists of a cascade of multiple-scale spatial contextual module and spatial-spectral fusion Transformer module. Specifically, the multiple-scale spatial contextual module performs three-level feature extraction to learn the spatial contextual structures of each spectral band from the RGB image. The spatial-spectral fusion Transformer module employs three parallel spatial-spectral united Transformers and one fusion Transformer to enhance the spatial and spectral consistency of the reconstructed spectral image. Parallel deployment of spatial-spectral united Transformers is beneficial to reduce the over-smooth effect caused by stacking too many Transformers. Therefore, the proposed network can better recover hyperspectral images with both fine spatial structures and accurate spectral signatures. Comprehensive experiments show that our method can achieve better reconstruction performance than the state-of-the-art methods.

Tuning IR-cut Filter for Illumination-aware Spectral Reconstruction from RGB

Optimal Selection of Representative Colors for Spectral Reflectance Reconstruction in a Multispectral Imaging System

Sequential Selection Of Representative Color Samples For Spectral Reflectance Reconstruction

Spectral Bidirectional Texture Function Reconstruction by Fusing Multiple-Color and Spectral Images.

Color Router-based Long-wave Infrared Multispectral Imaging

Improved Reflectance Reconstruction for Multispectral Imaging by Combining Different Techniques.

Improvement of Underwater Color Discriminative Ability by Multispectral Imaging

Compressive Single-Pixel Hyperspectral Imaging Using RGB Sensors

Multispectral Image Super-Resolution Via RGB Image Fusion and Radiometric Calibration

Hyperspectral image recovery based on fusion of coded aperture snapshot spectral imaging and RGB images by guided filtering

Representation Selective Coupling Via Token Sparsification for Multi-Spectral Object Re-Identification

Spectral Reconstruction Using an Iteratively Reweighted Regulated Model from Two Illumination Camera Responses

Spectral Reconstruction from RGB Imagery: A Potential Option for Infinite Spectral Data?

Training-Based Spectral Reconstruction from a Single RGB Image

Fast Multispectral Imaging by Spatial Pixel-Binning and Spectral Unmixing

Limitations of Data-Driven Spectral Reconstruction -- Optics-Aware Analysis and Mitigation

A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging

Spectral reconstruction from RGB image to hyperspectral image: Take the detection of glutamic acid index in beef as an example

Spec-NeRF: Multi-spectral Neural Radiance Fields

Multi-scale spatial-spectral Transformer for spectral reconstruction from RGB images

Estimating spectral reflectance in a spectral imaging system