Abstract:To solve the low spatial and/or temporal resolution problem which the conventional hyperspectral cameras often suffer from, spectral compressive imaging systems (SCI) have attracted more attention recently. Recovering a hyperspectral image (HSI) from its corresponding 2D coded image is an ill-posed inverse problem, and learning accurate prior from HSI and 2D coded image is essential to solve this inverse problem. Existing methods only use supervised networks that focus on learning generalized prior from training datasets, or only use unsupervised networks that focus on learning specific prior from 2D coded image, resulting in the inability to learn both generalized and specific priors. Also, when learning the priors, existing methods cannot simultaneously give consideration to both global and local scales, as well as both spatial and spectral dimensions. To cope with this problem, in this paper, we propose a Supervised-Unsupervised Combined Transformer Network (SUCTNet) composed by a supervised Spatio-spectral Transformer network (SSTNet) and an Unsupervised Multi-level Feature Refinement network (UMFRNet). Specifically, we first develop the SSTNet to learn generalized prior and obtain a preliminary HSI. In SSTNet, the proposed spatial encoding and spectral decoding network architecture enables it to simultaneously consider both spatial and spectral dimensions, and a proposed Global and Local Multi head Self Attention block (GL-MSA) enables it simultaneously to consider both global and local scales. Then, the preliminary HSI is fed into the proposed UMFRNet to learn specific prior and obtain the target HSI. In UMFRNet, a proposed multi-level feature refinement mechanism and the physical imaging model of SCI are used to improve reconstruction accuracy and generalization performance. Extensive experiments show that our method significantly outperforms state-of-the-art (SOTA) methods on simulated and real datasets. Codes will be available at https://github.com/Vzhouhan/SUCTNet .

Region-based Spectral-Spatial Mutual Induction Network for Hyperspectral Image Reconstruction

Hyperspectral Image Reconstruction from RGB Input Through Highlighting Intrinsic Properties

Reconstructing hyperspectral images from rgb inputs based on intrinsic image decomposition

Compressed Sensing Reconstruction of Hyperspectral Images Based on Spatial-spectral Multihypothesis Prediction

Compressed Sensing Reconstruction Of Hyperspectral Images Jointly Using Spatial Smoothing Feature And Spectral Correlation

Unsupervised Spatial-spectral Network Learning for Hyperspectral Compressive Snapshot Reconstruction

Spectral-wise Implicit Neural Representation for Hyperspectral Image Reconstruction

Hyperspectral Image Reconstruction Based on Improved Residual Dense Network

Implicit Neural Representation Learning for Hyperspectral Image Super-Resolution

Hyperspectral Compressive Snapshot Reconstruction via Coupled Low-Rank Subspace Representation and Self-Supervised Deep Network

Hyperspectral Reconstruction Using Auxiliary Rgb Learning From A Snapshot Image

Mask-Guided Spatial–Spectral MLP Network for High-Resolution Hyperspectral Image Reconstruction

Hyperspectral Image Reconstruction via Combinatorial Embedding of Cross-Channel Spatio-Spectral Clues

Supervised-unsupervised combined transformer for spectral compressive imaging reconstruction

Exploiting Frequency Correlation for Hyperspectral Image Reconstruction

Hyperspectral Compressive Image Reconstruction With Deep Tucker Decomposition and Spatial–Spectral Learning Network

Hyperspectral Image Reconstruction Via Patch Attention Driven Network

MAUN: Memory-Augmented Deep Unfolding Network for Hyperspectral Image Reconstruction

DS PN: A Two-Stage Direction-Aware Spectral-Spatial Perceptual Network for Hyperspectral Image Reconstruction

Learning Tensor Low-Rank Prior for Hyperspectral Image Reconstruction

Joint Spatial-Spectral Pattern Optimization and Hyperspectral Image Reconstruction