Abstract:Fusing hyperspectral image (HSI) and multispectral image (MSI) of high spatial resolution is typically utilized to obtain HSIs of high spatial resolution. However, the spatial quality of most existing methods is unsatisfactory due to the limited spatial resolution of an MSI. To further improve the spatial resolution of the fused HSI while keeping the spectral information well, we propose a new computational paradigm, named HyperFusion, which simultaneously fuses HSI, MSI, and panchromatic (PAN) image. To achieve this goal, we first establish two data fidelity terms based on a physical observation that HSI and MSI can be treated as degraded versions of the fused HSI. Consequently, the spatial and spectral information from HSI and MSI can be well preserved. To efficiently transfer the spatial details of PAN into the fused HSI while keeping the spectral information well, we further construct a prior constraint from PAN based on the structural similarity. Meanwhile, we impose another low-rank prior constraint on the coefficient matrix to accurately describe the latent characteristics of the HSI with high spatial resolution. By incorporating the aforementioned data fidelity terms and prior constraints, we finally formulate the objective as an optimization problem and utilize the alternative direction multiplier method to solve it efficiently. Comprehensive experiments on simulated and real datasets are carried out to demonstrate the superiority of HyperFusion over other state of the arts in terms of visual quality and quantitative analysis. We also adopt a simulated experiment of vegetation coverage index analysis to verify the effectiveness of HyperFusion in remote sensing applications.

Hyperspectral and Multispectral Image Fusion - From Model-Driven to Data-Driven.

Hyperspectral and Multispectral Image Fusion: when Model-driven Meet Data-driven Strategies

Unaligned Hyperspectral Image Fusion Via Registration and Interpolation Modeling

Recent Advances and New Guidelines on Hyperspectral and Multispectral Image Fusion

Hyperspectral and Multispectral Image Fusion Based on Deep Attention Network.

HYPERSPECTRAL AND MULTISPECTRAL IMAGE FUSION USING DUAL-SOURCE LOCALIZED DICTIONARY PAIR

Hyperspectral–Multispectral Image Fusion Enhancement Based on Deep Learning

Hyperspectral and Multispectral Image Fusion with Dual-Source Spatial-Spectral Dictionary.

Hyperspectral and Multispectral Image Fusion Based on Band Simulation.

Hyperspectral And Multispectral Image Fusion Based On Spectral Low Rank And Non-Local Spatial Similarities

HyperFusion: A Computational Approach for Hyperspectral, Multispectral, and Panchromatic Image Fusion

Hyperspectral and Multispectral Image Fusion Using Local Spatial-Spectral Dictionary Pair

Hyperspectral and Multispectral Image Fusion via Variational Tensor Subspace Decomposition

Hyperspectral and Multispectral Image Fusion Based on Local Low Rank and Coupled Spectral Unmixing

Unsupervised Hyperspectral and Multispectral Image Fusion with Deep Spectral-Spatial Collaborative Constraint

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net.

Spectral Unmixing Based Fusion Algorithm For Hyperspectral And Multi-Spectral Images

Hyperspectral and Multispectral Image Fusion Using the Conditional Denoising Diffusion Probabilistic Model

Hyperspectral and Multispectral Image Fusion Via Nonlocal Low-Rank Tensor Decomposition and Spectral Unmixing

A Novel Adversarial Based Hyperspectral and Multispectral Image Fusion