Abstract:The goal of pansharpening methods is to complement the spectral and spatial information contained in Multi-spectral (MS) and panchromatic (PAN) images to obtain the desired High-resolution multispectral (HRMS) image. The existing majority of pansharpening methods either extract feature information separately from the MS image and PAN image, or extract feature information after concatenating the MS image and PAN image. However, the entire extraction process lacks the utilization of complementary information and tends to generate redundant information, thereby leading to the loss of certain important information during the extraction process, which in turn affects the overall performance. In order to better utilize the complementary information between the MS image and PAN image and enhance the interpretability of the network, we propose the Deep Unfolding Convolutional-Dictionary Network (DUCD) for pansharpening in this paper. This network fully integrates complementary information between the MS image and PAN image to generate the final fused image. The entire network structure consists of two parts: The encoder and the decoder. In the encoder part of the network, we clarify the common and unique feature information between MS and PAN images by constructing an observation model. Simultaneously, we use the approximate gradient algorithm to continuously optimize the model and iteratively unfold it into a deep network structure. In the decoder part of the network, we concatenate the obtained common and specific information from MS and PAN images and pass them through convolutional and activation layers. Subsequently, they are input into the introduced Frequency Domain-based Transformer (FDT) module and an information-lossless inversible neural network(INN). This provides a more efficient method for establishing long-range dependency relationships between feature extraction and feature fusion. To demonstrate the effectiveness of our proposed method, we conduct extensive experiments on three benchmark datasets QB, GF2 and WV3. Experimental results show that our method outperforms the current SOTA Pansharpening methods in terms of performance.

Going Deeper with Densely Connected Convolutional Neural Networks for Multispectral Pansharpening

Efficient and Accurate Hyperspectral Pansharpening Using 3D VolumeNet and 2.5D Texture Transfer

SSETPAN: Spatial-Spectral Enhanced Transformer Based Network for Pansharpening

DuCoFPan: Dual-Condition Flow-based Network for Pan-sharpening

Multispectral Pan-sharpening via Dual-Channel Convolutional Network with Convolutional LSTM Based Hierarchical Spatial-Spectral Feature Fusion

Multispectral Pansharpening Based on Multisequence Convolutional Recurrent Neural Network

Convolutional LSTM-Based Hierarchical Feature Fusion for Multispectral Pan-Sharpening

A Triple-Double Convolutional Neural Network for Panchromatic Sharpening

MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network

Convolutional Neural Network for Pansharpening with Spatial Structure Enhancement Operator

Pansharpening via Detail Injection Based Convolutional Neural Networks

DMCSC: Deep Multisource Convolutional Sparse Coding Model for Pansharpening

Pan-Sharpening via Multiscale Dynamic Convolutional Neural Network

Detail Information Prior Net for Remote Sensing Image Pansharpening

Deep Multiscale Detail Networks for Multiband Spectral Image Sharpening

Transformer-based adaptive 3D residual CNN with sparse representation for PAN-sharpening of multispectral images

DUCD: Deep Unfolding Convolutional-Dictionary network for pansharpening remote sensing image

Pan-Sharpening with Customized Transformer and Invertible Neural Network

Pan-Sharpening Based On Parallel Pyramid Convolutional Neural Network

Laplacian pyramid networks: A new approach for multispectral pansharpening

SSCAConv: Self-Guided Spatial-Channel Adaptive Convolution for Image Fusion