Abstract:Due to the physical boundaries, fusing low spatial resolution hyperspectral (LrHSI) with high spatial resolution multispectral (HrMSI) images is a hot and promising area for obtaining hyperspectral that have high spatial-spectral resolution images (HrHSI). Effectively formulating the fundamental features of hyperspectral images (HSI), such as global spectral correlation, nonlocal spatial correlation, as well spatial-spectral correlation, is complex in HSI-MSI fusion. Moreover, the fusion process is highly affected by the degradation systems, where these systems are not known in real scenarios. To this end, in this article, we proposed a model-guided deep unfolded fusion network with non-local spatial-spectral priors (MGDuNLSS-net) that can maintain the essential features of the HSIs and implicitly estimates the degradation process in an adequate running time. Specifically, the proposed method is designed based on subspace representation in an iterative manner and unrolling its steps toward a deep network as an end-to-end framework. This approach contains two submodules, fusion [nonlocal spatial-spectral block (NLSSB)] and imaging system submodules. The former submodule is proposed to exploit the images' intrinsic characteristics to improve the preservation of spectral and spatial details. NLSSB contains two nonlocal self-similarity (NLSS) layers embedded between two bidirectional simple recurrent unit (BSRU) layers. The recurrent calculation, as well as refined components to maintain the global spectral correlation, are the light recurrence operation and highway network, while 3-D convolutions in the BSRU can retain the spatial-spectral correlation. The NLSS layer can efficiently and effectively model long-range spatial contexts, which is designed based on criss-cross attention. The later submodule is used to refine the prediction of the degradation process at any iteration via backprojecting the estimated fused image to the observed pair, which can ensure the good performance of fusion. Compared with state-of-the-art fusion approaches, three remote sensing datasets are used to validate the proposed approach's performance.

Unsupervised 3-D Tensor Subspace Decomposition Network for Spatial–Temporal–Spectral Fusion of Hyperspectral and Multispectral Images

Hyperspectral-multispectral image fusion using subspace decomposition and Elastic Net Regularization

Unsupervised Encoder-Decoder Network Under Spatial and Spectral Guidance for Hyperspectral and Multispectral Image Fusion

Unsupervised Hyperspectral and Multispectral Image Fusion with Deep Spectral-Spatial Collaborative Constraint

Unsupervised Hyperspectral and Multispectral Image Blind Fusion Based on Deep Tucker Decomposition Network with Spatial-Spectral Manifold Learning

Dual-Task Interactive Learning for Unsupervised Spatio-Temporal–Spectral Fusion of Remote Sensing Images

Hyperspectral and Multispectral Image Fusion via Variational Tensor Subspace Decomposition

Unsupervised Hybrid Network of Transformer and CNN for Blind Hyperspectral and Multispectral Image Fusion

Hyperspectral and Multispectral Image Fusion Via Nonlocal Low-Rank Tensor Decomposition and Spectral Unmixing

Unsupervised Deep Tensor Network for Hyperspectral–Multispectral Image Fusion

Unsupervised Blind Spectral-spatial Cross-super-resolution Network for HSI and MSI Fusion

Model-Guided Deep Unfolded Fusion Network with Nonlocal Spatial-Spectral Priors for Hyperspectral Image Super-Resolution.

Hyperspectral-Multispectral Image Fusion via Tensor Ring and Subspace Decompositions

Generating High-Resolution Hyperspectral Time Series Datasets Based on Unsupervised Spatial-Temporal-spectral Fusion Network Incorporating a Deep Prior

Nonlocal Coupled Tensor CP Decomposition for Hyperspectral and Multispectral Image Fusion.

Hyperspectral and Multispectral Image Fusion Based on Unsupervised Feature Mixing and Reconstruction Network.

Unified Cloud-Based Framework for Hyperspectral and Multispectral Image Fusion Incorporating Nonlocal Principles and Tensor Decomposition

Unsupervised Deep Adaptive Learning Spatial Reconstruction Network Based on Hyperspectral Data Fusion

Integrated Fusion Framework Based on Semicoupled Sparse Tensor Factorization for Spatio-Temporal-spectral Fusion of Remote Sensing Images.

Subspace-based coupled tensor decomposition for hyperspectral blind fusion

Spatial–Spectral-Graph-Regularized Low-Rank Tensor Decomposition for Multispectral and Hyperspectral Image Fusion