Abstract:The goal of pansharpening methods is to complement the spectral and spatial information contained in Multi-spectral (MS) and panchromatic (PAN) images to obtain the desired High-resolution multispectral (HRMS) image. The existing majority of pansharpening methods either extract feature information separately from the MS image and PAN image, or extract feature information after concatenating the MS image and PAN image. However, the entire extraction process lacks the utilization of complementary information and tends to generate redundant information, thereby leading to the loss of certain important information during the extraction process, which in turn affects the overall performance. In order to better utilize the complementary information between the MS image and PAN image and enhance the interpretability of the network, we propose the Deep Unfolding Convolutional-Dictionary Network (DUCD) for pansharpening in this paper. This network fully integrates complementary information between the MS image and PAN image to generate the final fused image. The entire network structure consists of two parts: The encoder and the decoder. In the encoder part of the network, we clarify the common and unique feature information between MS and PAN images by constructing an observation model. Simultaneously, we use the approximate gradient algorithm to continuously optimize the model and iteratively unfold it into a deep network structure. In the decoder part of the network, we concatenate the obtained common and specific information from MS and PAN images and pass them through convolutional and activation layers. Subsequently, they are input into the introduced Frequency Domain-based Transformer (FDT) module and an information-lossless inversible neural network(INN). This provides a more efficient method for establishing long-range dependency relationships between feature extraction and feature fusion. To demonstrate the effectiveness of our proposed method, we conduct extensive experiments on three benchmark datasets QB, GF2 and WV3. Experimental results show that our method outperforms the current SOTA Pansharpening methods in terms of performance.

Deep Self-Learning Network for Adaptive Pansharpening

Kernel-guided injection deep network for blind fusion of multispectral and panchromatic images

PanCSC-Net: A Model-Driven Deep Unfolding Method for Pansharpening

SDPNet: A Deep Network for Pan-Sharpening with Enhanced Information Representation

VO+Net: an Adaptive Approach Using Variational Optimization and Deep Learning for Panchromatic Sharpening

A Multi-Stage Progressive Pansharpening Network Based on Detail Injection with Redundancy Reduction

Self-Supervised Pansharpening Based on a Cycle-Consistent Generative Adversarial Network

Detail Injection-Based Deep Convolutional Neural Networks for Pansharpening

Unsupervised Pansharpening Based on Self-Attention Mechanism

Weighted Shallow-Deep Feature Fusion Network for Pansharpening.

Deep Gradient Projection Networks for Pan-sharpening

DUCD: Deep Unfolding Convolutional-Dictionary network for pansharpening remote sensing image

A deep unrolling pansharpening method based on spectral consistency and double spatial priors

Rethinking Pan-Sharpening Via Spectral-Band Modulation

A Unified Two-Stage Spatial and Spectral Network With Few-Shot Learning for Pan-Sharpening

An Efficient Pansharpening Approach Based on Spatial Information and Multi-Layer Fusion.

Multiscale Dual-Domain Guidance Network for Pan-Sharpening

Unsupervised Deep Learning-based Pansharpening with Jointly-Enhanced Spectral and Spatial Fidelity

Pannet: A Deep Network Architecture For Pan-Sharpening

Multiscale Spatial-Frequency Domain Dynamic Pansharpening of Remote Sensing Images Integrated With Wavelet Transform

Pansharpening by Combining Enhanced Spatial-Spectral Fidelity and Gradient-Domain Guided Filtering