Abstract:Pan-sharpening aims to super-resolve the low-resolution (LR) multispectral (MS) image under the guidance of a high-resolution (HR) panchromatic (PAN) image. Existing deep learning (DL)-based pan-sharpening methods usually adhere to a common philosophy of learning complementary information between MS and PAN images. Despite remarkable advances, few studies consider the band-private characteristics which differ greatly from band to band. An ideal MS image, however, is jointly determined by its diverse spectral bands, thus the accurate restoration of every band will benefit the pan-sharpening performance. In this work, we propose a novel yet effective solution to reconstruct the HRMS image by explicitly modulating every spectral band under the conditions of the PAN image. As a result, we design a spatially-adaptive spectral modulation network, dubbed SSMNet, which consists of three core designs: source-aware spectral modulator (SSM), cross-band information aggregation (CBIA) module, and cross-stage feature integration (CSFI) module. The first predicts a series of spatially-adaptive kernels to capture the local information of every spectral band. Followed by, the second is responsible for facilitating the information communication among various bands to guarantee continuous spectral representations. Furthermore, the third attends to integrate the cross-stage output features to produce the pan-sharpened result. In addition, we also introduce the histogram loss to constrain the band-wise distribution of the final fused products. Extensive experiments demonstrate that our SSMNet achieves favorable performance against other state-of-the-art (SOTA) methods on multiple satellite datasets. The code is available at https://github.com/ez4lionky/SSMNet/ .

Learning Deep Multiscale Local Dissimilarity Prior for Pansharpening.

Coarse-to-Fine Cross-Domain Learning Fusion Network for Pansharpening.

Cascadic Multireceptive Learning for Multispectral Pansharpening

Multiscale Dual-Domain Guidance Network for Pan-Sharpening

LDP-Net: an Unsupervised Pansharpening Network Based on Learnable Degradation Processes

Spatial and Spectral Extraction Network With Adaptive Feature Fusion for Pansharpening

SSML: Spectral-Spatial Mutual-Learning-Based Framework for Hyperspectral Pansharpening.

Pan-Sharpening Based on Multilevel Coupled Deep Network

DPDU-Net: Double Prior Deep Unrolling Network for Pansharpening

DCPNet: A Dual-Task Collaborative Promotion Network for Pansharpening

A deep unrolling pansharpening method based on spectral consistency and double spatial priors

Multispectral Pan-sharpening via Dual-Channel Convolutional Network with Convolutional LSTM Based Hierarchical Spatial-Spectral Feature Fusion

A Two-Stream Multiscale Deep Learning Architecture for Pan-Sharpening

PSMD-Net: A Novel Pan-Sharpening Method Based on a Multiscale Dense Network

Supervised-unsupervised Combined Deep Convolutional Neural Networks for High-Fidelity Pansharpening

Cross Spectral and Spatial Scale Non-local Attention-Based Unsupervised Pansharpening Network

A Multi-Scale Densely Deep Learning Method for Pansharpening.

A Unified Pansharpening Method with Structure Tensor Driven Spatial Consistency and Deep Plug-and-Play Priors

Rethinking Pan-Sharpening Via Spectral-Band Modulation

MSDRN: Pansharpening of Multispectral Images Via Multi-Scale Deep Residual Network.

Hyperspectral Pansharpening Using Deep Prior and Dual Attention Residual Network