Abstract:Pansharpening aims to produce a high spatial resolution multispectral (HRMS) image by combining a low spatial resolution multispectral (LRMS) image with a high-resolution panchromatic (PAN) image through a fusion process. Deep learning (DL)-based pansharpening methods have demonstrated impressive results in generating high-quality HRMS images. However, they suffer from a lack of interpretability due to their black-box network architectures. Recently, model-based deep unrolling networks have been proposed to improve the interpretability of networks. Among these approaches, the multisource convolutional sparse coding (MCSC)-based models stand out by effectively learning common and unique features from both LRMS and PAN images, showing promising results. As the LRMS image provides limited information in MCSC-based models, it can result in weak feature response and even lead to incorrect fusion outcomes. To address this issue, we propose a novel deep MCSC-based method that enhances the robustness and performance. Specifically, we build an optimization model that integrates MCSC with a degradation model and a deep prior, which can sufficiently capture the common information shared by the latent HRMS images and PAN images, thereby enabling the recovery of more accurate spectral information. To optimize the proposed model, we adopt an iterative optimization strategy that unfolds the iterative solution into networks. Moreover, we propose an enhanced version of our method that utilizes multiscale dictionaries to capture common and unique features at different scales, thereby facilitating the extraction of more abundant spectral and spatial details. We evaluate the effectiveness of our proposed method on multiple benchmark datasets. Experiment results demonstrate its effectiveness in improving the robustness and performance of MCSC-based models.

MSCSCformer: Multiscale Convolutional Sparse Coding-Based Transformer for Pansharpening

MSCSCformer: Multi-scale Convolutional Sparse Coding-based Transformer for Pansharpening

SSETPAN: Spatial-Spectral Enhanced Transformer Based Network for Pansharpening

Efficient and Accurate Hyperspectral Pansharpening Using 3D VolumeNet and 2.5D Texture Transfer

DMCSC: Deep Multisource Convolutional Sparse Coding Model for Pansharpening

Multiscale 3-D-2-D Mixed CNN and Lightweight Attention-Free Transformer for Hyperspectral and LiDAR Classification

Multispectral Pan-sharpening via Dual-Channel Convolutional Network with Convolutional LSTM Based Hierarchical Spatial-Spectral Feature Fusion

Transformer-Based Dual-Branch Multiscale Fusion Network for Pan-Sharpening Remote Sensing Images

MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network

PanFormer: a Transformer Based Model for Pan-sharpening

STCP: Synergistic Transformer and Convolutional Neural Network for Pansharpening

Local-Global Based High-Resolution Spatial-Spectral Representation Network for Pansharpening

MSCC-ViT:A Multiscale Visual-Transformer Network Using Convolution Crossing Attention for Hyperspectral Unmixing

Convolutional LSTM-Based Hierarchical Feature Fusion for Multispectral Pan-Sharpening

Going Deeper with Densely Connected Convolutional Neural Networks for Multispectral Pansharpening

CMT: Cross Modulation Transformer with Hybrid Loss for Pansharpening

Multispectral Pansharpening Based on Multisequence Convolutional Recurrent Neural Network

SSCAConv: Self-Guided Spatial-Channel Adaptive Convolution for Image Fusion

Transformer-based dual path cross fusion for pansharpening remote sensing images

SSconv

Multiscale Spatial-Frequency Domain Dynamic Pansharpening of Remote Sensing Images Integrated With Wavelet Transform