Abstract:Pan-sharpening aims to obtain high resolution multispectral (HRMS) images by integrating the information in the panchromatic and multispectral images. Existing pan-sharpening methods have demonstrated impressive sharpening performance. However, these methods inherently overlook the complementary characteristics and interaction between diverse source images, resulting in sharpened outcomes accompanied by distortion. To solve the above problems, we construct a novel cross fusion model with fine-grained detail reconstruction from the perspective of frequency-domain. The motivation of the model is twofold: (1) to reconstruct spatial detail representations from diverse source images, laying the foundation for the generation of fine details in the subsequent fused images; and (2) to enhance the interaction between diverse source features during the fusion process in order to attain high-fidelity fusion outcomes. Based on the theoretical model, we develop a frequency-spectral dual domain cross fusion network (CF2N) utilizing the deep learning technique. Consequently, the CF2N consist of two main stages, namely frequency-domain dominated detail reconstruction (FD2R) and frequency-spectral cross fusion (FSCF). Specifically, a more reasonable reconstruction of fine frequency details in HRMS can be achieved by performing adaptive weighted fusion of frequency details in the FD2R stage. Furthermore, the FSCF module, which seamlessly integrates frequency- and spectral-domain details in a highly interactive cross fusion manner. As a result, the CF2N possesses the capability to attain high frequency-spectral fidelity results with excellent interpretability. Extensive experiments show the superior performance of ours over state of the art, while maintaining high efficiency. All implementations of this work will be published at our website.

A self-supervised remote sensing image fusion framework with dual-stage self-learning and spectral super-resolution injection

Multispectral Image Super-Resolution Via RGB Image Fusion and Radiometric Calibration

DuCoFPan: Dual-Condition Flow-based Network for Pan-sharpening

Detail Information Prior Net for Remote Sensing Image Pansharpening

Pan-sharpening Method Based on Unsupervised and Divide-and-conquer Strategies

Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion

Self-Supervised Pansharpening Based on a Cycle-Consistent Generative Adversarial Network

Real-time and effective pan-sharpening for remote sensing using multi-scale fusion network

Self-supervised spectral super-resolution for a fast hyperspectral and multispectral image fusion

A novel cross fusion model with fine-grained detail reconstruction for remote sensing image pan-sharpening

A Multi-Stage Progressive Pansharpening Network Based on Detail Injection with Redundancy Reduction

Remote Sensing Image Fusion Based on Two-stream Fusion Network

Unsupervised Pansharpening Based on Double-Cycle Consistency

Spatial and Spectral Extraction Network With Adaptive Feature Fusion for Pansharpening

Fusion of Hyperspectral and Multispectral Images: A Novel Framework Based on Generalization of Pan-Sharpening Methods

Panchromatic and multispectral image fusion for remote sensing and earth observation: Concepts, taxonomy, literature review, evaluation methodologies and challenges ahead

MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network

A Unified Two-Stage Spatial and Spectral Network With Few-Shot Learning for Pan-Sharpening

Unsupervised Pansharpening Based on Self-Attention Mechanism

An Unsupervised CNN-Based Pansharpening Framework with Spectral-Spatial Fidelity Balance

Multispectral Pan-sharpening via Dual-Channel Convolutional Network with Convolutional LSTM Based Hierarchical Spatial-Spectral Feature Fusion