Abstract:Pan-sharpening aims to super-resolve the low-resolution (LR) multispectral (MS) image under the guidance of a high-resolution (HR) panchromatic (PAN) image. Existing deep learning (DL)-based pan-sharpening methods usually adhere to a common philosophy of learning complementary information between MS and PAN images. Despite remarkable advances, few studies consider the band-private characteristics which differ greatly from band to band. An ideal MS image, however, is jointly determined by its diverse spectral bands, thus the accurate restoration of every band will benefit the pan-sharpening performance. In this work, we propose a novel yet effective solution to reconstruct the HRMS image by explicitly modulating every spectral band under the conditions of the PAN image. As a result, we design a spatially-adaptive spectral modulation network, dubbed SSMNet, which consists of three core designs: source-aware spectral modulator (SSM), cross-band information aggregation (CBIA) module, and cross-stage feature integration (CSFI) module. The first predicts a series of spatially-adaptive kernels to capture the local information of every spectral band. Followed by, the second is responsible for facilitating the information communication among various bands to guarantee continuous spectral representations. Furthermore, the third attends to integrate the cross-stage output features to produce the pan-sharpened result. In addition, we also introduce the histogram loss to constrain the band-wise distribution of the final fused products. Extensive experiments demonstrate that our SSMNet achieves favorable performance against other state-of-the-art (SOTA) methods on multiple satellite datasets. The code is available at https://github.com/ez4lionky/SSMNet/ .

Remote Sensing Image Fusion Using Multiscale Mapped Ls-Svm

SSETPAN: Spatial-Spectral Enhanced Transformer Based Network for Pansharpening

Transformer-Based Dual-Branch Multiscale Fusion Network for Pan-Sharpening Remote Sensing Images

Multispectral And Panchromatic Images Fusion Based On Nsct And Gs Transform

Multispectral and Panchromatic Image Fusion Based on Improved Bilateral Filter

Fusion Of Multispectral And Panchromatic Images Based On Transferable Parameters

The Study on Image Fusion for Medium and High Spatial Resolution Remote Sensing Images

Fusion of Multispectral and Panchromatic Images Based on a Novel Inter-Band Structure Model

Multiscale Spatial-Frequency Domain Dynamic Pansharpening of Remote Sensing Images Integrated With Wavelet Transform

Fusion of multispectral and panchromatic images based on support value transform and adaptive principal component analysis

PAN-Guided Band-Aware Multi-Spectral Feature Enhancement for Pan-Sharpening

A Two-Stage Pansharpening Method for the Fusion of Remote-Sensing Images

Detail Information Prior Net for Remote Sensing Image Pansharpening

Panchromatic Sharpening of Remote Sensing Images Using a Multi-scale Approach

Rethinking Pan-Sharpening Via Spectral-Band Modulation

Pan-Sharpening by Multilevel Interband Structure Modeling.

Real-time and effective pan-sharpening for remote sensing using multi-scale fusion network

Fusion of Hyperspectral and Multispectral Images: A Novel Framework Based on Generalization of Pan-Sharpening Methods

The Fusion of Panchromatic and Multispectral Remote Sensing Images Via Tensor-Based Sparse Modeling and Hyper-Laplacian Prior

Multispectral and Panchromatic Image Fusion Based on Unsubsampled Contourlet Transform

MDSCNN: Remote Sensing Image Spatial–Spectral Fusion Method via Multi-Scale Dual-Stream Convolutional Neural Network