Abstract:Pan-sharpening, as one of the most commonly used techniques in remote sensing systems, aims to inject spatial details from panchromatic images into multispectral images (MS) to obtain high-resolution multispectral images. Since deep learning has received widespread attention because of its powerful fitting ability and efficient feature extraction, a variety of pan-sharpening methods have been proposed to achieve remarkable performance. However, current pan-sharpening methods usually require the paired panchromatic (PAN) and MS images as input, which limits their usage in some scenarios. To address this issue, in this paper we observe that the spatial details from PAN images are mainly high-frequency cues, i.e., the edges reflect the contour of input PAN images. This motivates us to develop a PAN-agnostic representation to store some base edges, so as to compose the contour for the corresponding PAN image via them. As a result, we can perform the pan-sharpening task with only the MS image when inference. To this end, a memory-based network is adapted to extract and memorize the spatial details during the training phase and is used to replace the process of obtaining spatial information from PAN images when inference, which is called Memory-based Spatial Details Network (MSDN). Finally, we integrate the proposed MSDN module into the existing deep learning-based pan-sharpening methods to achieve an end-to-end pan-sharpening network. With extensive experiments on the Gaofen1 and WorldView-4 satellites, we verify that our method constructs good spatial details without PAN images and achieves the best performance. The code is available at <a class="link-external link-https" href="https://github.com/Zhao-Tian-yi/Learning-to-Pan-sharpening-with-Memories-of-Spatial-Details.git" rel="external noopener nofollow">this https URL</a>.

Sparse Representation-Based Detail-Injection Method for Pan-Sharpening

DuCoFPan: Dual-Condition Flow-based Network for Pan-sharpening

SSETPAN: Spatial-Spectral Enhanced Transformer Based Network for Pansharpening

Domain-specific knowledge-driven pan-sharpening algorithm

Detail Information Prior Net for Remote Sensing Image Pansharpening

A Multi-Stage Progressive Pansharpening Network Based on Detail Injection with Redundancy Reduction

PAN-Guided Band-Aware Multi-Spectral Feature Enhancement for Pan-Sharpening

A pansharpening method combining iterative filtering and NSST-NSML-PAPCNN to optimize spatial detail extraction and injection

Pansharpening by Combining Enhanced Spatial-Spectral Fidelity and Gradient-Domain Guided Filtering

Weighted manifold regularized sparse representation of featured injected details for pansharpening

Pansharpening via Detail Injection Based Convolutional Neural Networks

Learning Interpolation Via Regional Map for Pan-Sharpening

Sparse Representation over Shared Coefficients in Multispectral Pansharpening

Learning to Pan-sharpening with Memories of Spatial Details

Multispectral Image Pan-Sharpening Guided by Component Substitution Model

Pan-sharpening Based on Common Saliency Feature Analysis and Multiscale Spatial Information Extraction for Multiple Remote Sensing Images

A deep unrolling pansharpening method based on spectral consistency and double spatial priors

Pan-Sharpening Based On Parallel Pyramid Convolutional Neural Network

Pan-Sharpening via Multiscale Dynamic Convolutional Neural Network

Detail-Injection-Based Multiscale Asymmetric Residual Network for Pansharpening

Spatial-Frequency Domain Information Integration for Pan-Sharpening