Abstract:Fusing hyperspectral image (HSI) and multispectral image (MSI) of high spatial resolution is typically utilized to obtain HSIs of high spatial resolution. However, the spatial quality of most existing methods is unsatisfactory due to the limited spatial resolution of an MSI. To further improve the spatial resolution of the fused HSI while keeping the spectral information well, we propose a new computational paradigm, named HyperFusion, which simultaneously fuses HSI, MSI, and panchromatic (PAN) image. To achieve this goal, we first establish two data fidelity terms based on a physical observation that HSI and MSI can be treated as degraded versions of the fused HSI. Consequently, the spatial and spectral information from HSI and MSI can be well preserved. To efficiently transfer the spatial details of PAN into the fused HSI while keeping the spectral information well, we further construct a prior constraint from PAN based on the structural similarity. Meanwhile, we impose another low-rank prior constraint on the coefficient matrix to accurately describe the latent characteristics of the HSI with high spatial resolution. By incorporating the aforementioned data fidelity terms and prior constraints, we finally formulate the objective as an optimization problem and utilize the alternative direction multiplier method to solve it efficiently. Comprehensive experiments on simulated and real datasets are carried out to demonstrate the superiority of HyperFusion over other state of the arts in terms of visual quality and quantitative analysis. We also adopt a simulated experiment of vegetation coverage index analysis to verify the effectiveness of HyperFusion in remote sensing applications.

Hyperspectral and Multispectral Image Fusion: when Model-driven Meet Data-driven Strategies

Hyperspectral and Multispectral Image Fusion - From Model-Driven to Data-Driven.

Unaligned Hyperspectral Image Fusion Via Registration and Interpolation Modeling

Hyperspectral and Multispectral Image Fusion Based on Deep Attention Network.

HYPERSPECTRAL AND MULTISPECTRAL IMAGE FUSION USING DUAL-SOURCE LOCALIZED DICTIONARY PAIR

Recent Advances and New Guidelines on Hyperspectral and Multispectral Image Fusion

Hyperspectral and Multispectral Image Fusion with Dual-Source Spatial-Spectral Dictionary.

Unsupervised Hyperspectral and Multispectral Image Fusion with Deep Spectral-Spatial Collaborative Constraint

HyperFusion: A Computational Approach for Hyperspectral, Multispectral, and Panchromatic Image Fusion

Hyperspectral–Multispectral Image Fusion Enhancement Based on Deep Learning

Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net.

Hyperspectral and Multispectral Image Fusion via Deep Two-Branches Convolutional Neural Network

Multi-Sensor Image Fusion: A Survey of the State of the Art

Hyperspectral and Multispectral Image Fusion Using Local Spatial-Spectral Dictionary Pair

Hyperspectral and Multispectral Image Fusion Via Nonlocal Low-Rank Tensor Decomposition and Spectral Unmixing

Hyperspectral and Multispectral Image Fusion Based on Band Simulation.

Hyperspectral and Multispectral Image Fusion Based on Local Low Rank and Coupled Spectral Unmixing

Hyperspectral And Multispectral Image Fusion Using Collaborative Representation With Local Adaptive Dictionary Pair

Hyperspectral And Multispectral Image Fusion Based On Spectral Low Rank And Non-Local Spatial Similarities

Hyperspectral and Multispectral Image Fusion via Variational Tensor Subspace Decomposition

Unsupervised Encoder-Decoder Network Under Spatial and Spectral Guidance for Hyperspectral and Multispectral Image Fusion