Abstract:To solve the low spatial and/or temporal resolution problem which the conventional hyperspectral cameras often suffer from, spectral compressive imaging systems (SCI) have attracted more attention recently. Recovering a hyperspectral image (HSI) from its corresponding 2D coded image is an ill-posed inverse problem, and learning accurate prior from HSI and 2D coded image is essential to solve this inverse problem. Existing methods only use supervised networks that focus on learning generalized prior from training datasets, or only use unsupervised networks that focus on learning specific prior from 2D coded image, resulting in the inability to learn both generalized and specific priors. Also, when learning the priors, existing methods cannot simultaneously give consideration to both global and local scales, as well as both spatial and spectral dimensions. To cope with this problem, in this paper, we propose a Supervised-Unsupervised Combined Transformer Network (SUCTNet) composed by a supervised Spatio-spectral Transformer network (SSTNet) and an Unsupervised Multi-level Feature Refinement network (UMFRNet). Specifically, we first develop the SSTNet to learn generalized prior and obtain a preliminary HSI. In SSTNet, the proposed spatial encoding and spectral decoding network architecture enables it to simultaneously consider both spatial and spectral dimensions, and a proposed Global and Local Multi head Self Attention block (GL-MSA) enables it simultaneously to consider both global and local scales. Then, the preliminary HSI is fed into the proposed UMFRNet to learn specific prior and obtain the target HSI. In UMFRNet, a proposed multi-level feature refinement mechanism and the physical imaging model of SCI are used to improve reconstruction accuracy and generalization performance. Extensive experiments show that our method significantly outperforms state-of-the-art (SOTA) methods on simulated and real datasets. Codes will be available at https://github.com/Vzhouhan/SUCTNet .

Deep Unfolding 3D Non-Local Transformer Network for Hyperspectral Snapshot Compressive Imaging

Convolutional Neural Network Based Computational Imaging Spectroscopy

Hybrid Sparse Transformer and Wavelet Fusion-Based Deep Unfolding Network for Hyperspectral Snapshot Compressive Imaging

Efficient and Accurate Hyperspectral Pansharpening Using 3D VolumeNet and 2.5D Texture Transfer

Pixel Adaptive Deep Unfolding Transformer for Hyperspectral Image Reconstruction

MAUN: Memory-Augmented Deep Unfolding Network for Hyperspectral Image Reconstruction

Hyperspectral Snapshot Compressive Imaging with Non-Local Spatial-Spectral Residual Network

Proximal Gradient Descent Unfolding Dense-spatial Spectral-attention Transformer for Compressive Spectral Imaging

Advancing Hyperspectral and Multispectral Image Fusion: An Information-Aware Transformer-Based Unfolding Network

Adaptive Step-size Perception Unfolding Network with Non-local Hybrid Attention for Hyperspectral Image Reconstruction

Deep Unfolding Network Enhanced by Transformer Priors for Unregistered Hyperspectral and Multispectral Image Fusion

Hyperspectral Compressive Image Reconstruction With Deep Tucker Decomposition and Spatial–Spectral Learning Network

Unsupervised Spatial-spectral Network Learning for Hyperspectral Compressive Snapshot Reconstruction

A Deep Unfolding Network for Multispectral and Hyperspectral Image Fusion

Latent Diffusion Prior Enhanced Deep Unfolding for Snapshot Spectral Compressive Imaging

Exploiting sparsity of hyperspectral image: A novel approach for compressive hyperspectral image reconstruction using deep learning

Hyperspectral Image Super-resolution via Knowledge-Driven Deep Unrolling and Transformer Embedded Convolutional Recurrent Neural Network

Supervised-unsupervised combined transformer for spectral compressive imaging reconstruction

Pixel-Based Long-Wave Infrared Spectral Image Reconstruction Using a Hierarchical Spectral Transformer

Dense Deep Unfolding Network with 3D-CNN Prior for Snapshot Compressive Imaging

Degradation-Aware Unfolding Half-Shuffle Transformer for Spectral Compressive Imaging