Abstract:Multispectral and hyperspectral image fusion (MS/HS fusion) aims to fuse a high-resolution multispectral (HrMS) and a low-resolution hyperspectral (LrHS) images to generate a high-resolution hyperspectral (HrHS) image, which has become one of the most commonly addressed problems for hyperspectral image processing. In this paper, we specifically designed a network architecture for the MS/HS fusion task, called MHF-net, which not only contains clear interpretability, but also reasonably embeds the well studied linear mapping that links the HrHS image to HrMS and LrHS images. In particular, we first construct an MS/HS fusion model which merges the generalization models of low-resolution images and the low-rankness prior knowledge of HrHS image into a concise formulation, and then we build the proposed network by unfolding the proximal gradient algorithm for solving the proposed model. As a result of the careful design for the model and algorithm, all the fundamental modules in MHF-net have clear physical meanings and are thus easily interpretable. This not only greatly facilitates an easy intuitive observation and analysis on what happens inside the network, but also leads to its good generalization capability. Based on the architecture of MHF-net, we further design two deep learning regimes for two general cases in practice: consistent MHF-net and blind MHF-net. The former is suitable in the case that spectral and spatial responses of training and testing data are consistent, just as considered in most of the pervious general supervised MS/HS fusion researches. The latter ensures a good generalization in mismatch cases of spectral and spatial responses in training and testing data, and even across different sensors, which is generally considered to be a challenging issue for general supervised MS/HS fusion methods. Experimental results on simulated and real data substantiate the superiority of our method both visually and quantitatively as compared with state-of-the-art methods along this line of research.

UMAG-Net: A New Unsupervised Multiattention-Guided Network for Hyperspectral and Multispectral Image Fusion

Hyperspectral and Multispectral Image Fusion Based on Deep Attention Network.

A Self-Supervised Spaceborne Multispectral and Hyperspectral Image Fusion Unrolling Network

Hyperspectral and Multispectral Image Fusion Based on Unsupervised Feature Mixing and Reconstruction Network.

Unsupervised Hyperspectral and Multispectral Image Fusion Based on Deep Adaptive Attention Network

UMTF-Net: An Unsupervised Multiscale Transformer Fusion Network for Hyperspectral and Multispectral Image Fusion

An Unmixing-Based Multi-Attention GAN for Unsupervised Hyperspectral and Multispectral Image Fusion

SSAU-Net: A Spectral–Spatial Attention-Based U-Net for Hyperspectral Image Fusion

Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net.

Hyperspectral and multispectral image fusion via residual selective kernel attention-based U-net

HyperNet: A Deep Network for Hyperspectral, Multispectral, and Panchromatic Image Fusion

MHF-Net: an Interpretable Deep Network for Multispectral and Hyperspectral Image Fusion

A Multiresolution Details Enhanced Attentive Dual-UNet for Hyperspectral and Multispectral Image Fusion

Unsupervised Encoder-Decoder Network Under Spatial and Spectral Guidance for Hyperspectral and Multispectral Image Fusion

Unsupervised Hyperspectral and Multispectral Image Fusion with Deep Spectral-Spatial Collaborative Constraint

Deep Unsupervised Blind Hyperspectral and Multispectral Data Fusion

FusionNet: An Unsupervised Convolutional Variational Network for Hyperspectral and Multispectral Image Fusion

A Deep Unfolding Network for Multispectral and Hyperspectral Image Fusion

Unsupervised Hybrid Network of Transformer and CNN for Blind Hyperspectral and Multispectral Image Fusion

Learning a Coupled Multilinear Network for Unsupervised Hyperspectral-Multispectral Image Fusion

Unsupervised Dynamic Convolutional Neural Network Model for Hyperspectral and Multispectral Image Fusion