Abstract:Pan-sharpening aims to super-resolve the low-resolution (LR) multispectral (MS) image under the guidance of a high-resolution (HR) panchromatic (PAN) image. Existing deep learning (DL)-based pan-sharpening methods usually adhere to a common philosophy of learning complementary information between MS and PAN images. Despite remarkable advances, few studies consider the band-private characteristics which differ greatly from band to band. An ideal MS image, however, is jointly determined by its diverse spectral bands, thus the accurate restoration of every band will benefit the pan-sharpening performance. In this work, we propose a novel yet effective solution to reconstruct the HRMS image by explicitly modulating every spectral band under the conditions of the PAN image. As a result, we design a spatially-adaptive spectral modulation network, dubbed SSMNet, which consists of three core designs: source-aware spectral modulator (SSM), cross-band information aggregation (CBIA) module, and cross-stage feature integration (CSFI) module. The first predicts a series of spatially-adaptive kernels to capture the local information of every spectral band. Followed by, the second is responsible for facilitating the information communication among various bands to guarantee continuous spectral representations. Furthermore, the third attends to integrate the cross-stage output features to produce the pan-sharpened result. In addition, we also introduce the histogram loss to constrain the band-wise distribution of the final fused products. Extensive experiments demonstrate that our SSMNet achieves favorable performance against other state-of-the-art (SOTA) methods on multiple satellite datasets. The code is available at https://github.com/ez4lionky/SSMNet/ .

Combining Component Substitution and Multiresolution Analysis: A Novel Generalized BDSD Pansharpening Algorithm

A New Pansharpening Method Using Multistage Joint Bilateral Filtering

Pansharpening Based on Joint Gaussian Guided Upsampling.

Hyperspectral Pansharpening Based on Intrinsic Image Decomposition and Weighted Least Squares Filter.

Spectral Diversity Enhancement for Pansharpening

An Effective Pansharpening Method Based on Guided Filtering

Rethinking Pan-Sharpening Via Spectral-Band Modulation

Pansharpening by Combining Enhanced Spatial-Spectral Fidelity and Gradient-Domain Guided Filtering

A New Hyperspectral Pansharpening Method with Intrisic Image Decomposition.

A Two-Stage Pansharpening Method for the Fusion of Remote-Sensing Images

Hyperspectral Pansharpening Via Local Intensity Component and Local Injection Gain Estimation

A Novel Scale-Aware Pansharpening Method

A Joint Optimization Based Pansharpening via Subpixel-Shift Decomposition

Pansharpening Based On An Improved Ratio Enhancement

Saliency Analysis and Gaussian Mixture Model-Based Detail Extraction Algorithm for Hyperspectral Pansharpening

Pansharpening Based on Fuzzy Logic and Edge Activity

Spectral-Driven Pansharpening Using Adaptive Image Segmentation To Reduce Color Distortion

A Pan-Sharpening Method Based on the ADMM Algorithm

Enhanced Back-Projection As Postprocessing for Pansharpening

Multispectral Image Pan-Sharpening Guided by Component Substitution Model

A Survey of Pansharpening Methods with A New Band-Decoupled Variational Model