Abstract:To solve the low spatial and/or temporal resolution problem which the conventional hyperspectral cameras often suffer from, spectral compressive imaging systems (SCI) have attracted more attention recently. Recovering a hyperspectral image (HSI) from its corresponding 2D coded image is an ill-posed inverse problem, and learning accurate prior from HSI and 2D coded image is essential to solve this inverse problem. Existing methods only use supervised networks that focus on learning generalized prior from training datasets, or only use unsupervised networks that focus on learning specific prior from 2D coded image, resulting in the inability to learn both generalized and specific priors. Also, when learning the priors, existing methods cannot simultaneously give consideration to both global and local scales, as well as both spatial and spectral dimensions. To cope with this problem, in this paper, we propose a Supervised-Unsupervised Combined Transformer Network (SUCTNet) composed by a supervised Spatio-spectral Transformer network (SSTNet) and an Unsupervised Multi-level Feature Refinement network (UMFRNet). Specifically, we first develop the SSTNet to learn generalized prior and obtain a preliminary HSI. In SSTNet, the proposed spatial encoding and spectral decoding network architecture enables it to simultaneously consider both spatial and spectral dimensions, and a proposed Global and Local Multi head Self Attention block (GL-MSA) enables it simultaneously to consider both global and local scales. Then, the preliminary HSI is fed into the proposed UMFRNet to learn specific prior and obtain the target HSI. In UMFRNet, a proposed multi-level feature refinement mechanism and the physical imaging model of SCI are used to improve reconstruction accuracy and generalization performance. Extensive experiments show that our method significantly outperforms state-of-the-art (SOTA) methods on simulated and real datasets. Codes will be available at https://github.com/Vzhouhan/SUCTNet .

MAUN: Memory-Augmented Deep Unfolding Network for Hyperspectral Image Reconstruction

A Self-Supervised Spaceborne Multispectral and Hyperspectral Image Fusion Unrolling Network

Hybrid Sparse Transformer and Wavelet Fusion-Based Deep Unfolding Network for Hyperspectral Snapshot Compressive Imaging

A Deep Unfolding Network Based on Intrinsic Image Decomposition for Pansharpening

Pixel Adaptive Deep Unfolding Transformer for Hyperspectral Image Reconstruction

A Deep Unfolding Network for Multispectral and Hyperspectral Image Fusion

Multi-Channel Representation Learning Enhanced Unfolding Multi-Scale Compressed Sensing Network for High Quality Image Reconstruction

Unsupervised Spatial-spectral Network Learning for Hyperspectral Compressive Snapshot Reconstruction

HerosNet: Hyperspectral Explicable Reconstruction and Optimal Sampling Deep Network for Snapshot Compressive Imaging

Adaptive Step-size Perception Unfolding Network with Non-local Hybrid Attention for Hyperspectral Image Reconstruction

Advancing Hyperspectral and Multispectral Image Fusion: An Information-Aware Transformer-Based Unfolding Network

Deep Unfolding Network Enhanced by Transformer Priors for Unregistered Hyperspectral and Multispectral Image Fusion

Supervised-unsupervised combined transformer for spectral compressive imaging reconstruction

Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction

Mask-Guided Spatial–Spectral MLP Network for High-Resolution Hyperspectral Image Reconstruction

UMAG-Net: A New Unsupervised Multiattention-Guided Network for Hyperspectral and Multispectral Image Fusion

Dual Hyperspectral Mamba for Efficient Spectral Compressive Imaging

Hyperspectral Image Super-resolution via Knowledge-Driven Deep Unrolling and Transformer Embedded Convolutional Recurrent Neural Network

Memory-Augmented Deep Unfolding Network for Compressive Sensing

Hyperspectral Compressive Image Reconstruction With Deep Tucker Decomposition and Spatial–Spectral Learning Network

Deep Unsupervised Blind Hyperspectral and Multispectral Data Fusion