Abstract:Due to the physical boundaries, fusing low spatial resolution hyperspectral (LrHSI) with high spatial resolution multispectral (HrMSI) images is a hot and promising area for obtaining hyperspectral that have high spatial-spectral resolution images (HrHSI). Effectively formulating the fundamental features of hyperspectral images (HSI), such as global spectral correlation, nonlocal spatial correlation, as well spatial-spectral correlation, is complex in HSI-MSI fusion. Moreover, the fusion process is highly affected by the degradation systems, where these systems are not known in real scenarios. To this end, in this article, we proposed a model-guided deep unfolded fusion network with non-local spatial-spectral priors (MGDuNLSS-net) that can maintain the essential features of the HSIs and implicitly estimates the degradation process in an adequate running time. Specifically, the proposed method is designed based on subspace representation in an iterative manner and unrolling its steps toward a deep network as an end-to-end framework. This approach contains two submodules, fusion [nonlocal spatial-spectral block (NLSSB)] and imaging system submodules. The former submodule is proposed to exploit the images' intrinsic characteristics to improve the preservation of spectral and spatial details. NLSSB contains two nonlocal self-similarity (NLSS) layers embedded between two bidirectional simple recurrent unit (BSRU) layers. The recurrent calculation, as well as refined components to maintain the global spectral correlation, are the light recurrence operation and highway network, while 3-D convolutions in the BSRU can retain the spatial-spectral correlation. The NLSS layer can efficiently and effectively model long-range spatial contexts, which is designed based on criss-cross attention. The later submodule is used to refine the prediction of the degradation process at any iteration via backprojecting the estimated fused image to the observed pair, which can ensure the good performance of fusion. Compared with state-of-the-art fusion approaches, three remote sensing datasets are used to validate the proposed approach's performance.

Unsupervised Encoder-Decoder Network Under Spatial and Spectral Guidance for Hyperspectral and Multispectral Image Fusion

Unsupervised Hyperspectral and Multispectral Image Fusion with Deep Spectral-Spatial Collaborative Constraint

Unsupervised Blind Spectral-spatial Cross-super-resolution Network for HSI and MSI Fusion

HYPERSPECTRAL AND MULTISPECTRAL IMAGE FUSION USING DUAL-SOURCE LOCALIZED DICTIONARY PAIR

Hyperspectral and Multispectral Image Fusion Based on Unsupervised Feature Mixing and Reconstruction Network.

Spatial Resolution Enhancement of Remote Sensing Hyperspectral Images with Localized Spatial-Spectral Dictionary Pair

A Multiresolution Details Enhanced Attentive Dual-UNet for Hyperspectral and Multispectral Image Fusion

U-Shape Spectral-Transformer for Robust Fusion Based Hyperspectral Super-Resolution

Fusion of Hyperspectral-Multispectral images joining Spatial-Spectral Dual-Dictionary and structured sparse Low-rank representation

SSAU-Net: A Spectral–Spatial Attention-Based U-Net for Hyperspectral Image Fusion

Spatial-Spectral Joint Correction Network for Hyperspectral and Multispectral Image Fusion

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

Multispectral and Hyperspectral Image Fusion with Spatial-Spectral Sparse Representation

Unsupervised Hybrid Network of Transformer and CNN for Blind Hyperspectral and Multispectral Image Fusion

Unsupervised 3-D Tensor Subspace Decomposition Network for Spatial–Temporal–Spectral Fusion of Hyperspectral and Multispectral Images

Hyperspectral and Multispectral Image Fusion Using Local Spatial-Spectral Dictionary Pair

Universal high spatial resolution hyperspectral imaging using hybrid-resolution image fusion

Model-Guided Deep Unfolded Fusion Network with Nonlocal Spatial-Spectral Priors for Hyperspectral Image Super-Resolution.

Deep Unsupervised Blind Hyperspectral and Multispectral Data Fusion

Deep unfolding Network for Hyperspectral Image Super-Resolution with Automatic Exposure Correction

Unsupervised Deep Tensor Network for Hyperspectral–Multispectral Image Fusion