Abstract:Pansharpening is a crucial task in remote sensing, enabling the generation of high-resolution multispectral images by fusing low-resolution multispectral data with high-resolution panchromatic images. This paper provides a comprehensive analysis of traditional and deep learning-based pansharpening methods. While state-of-the-art deep learning methods have significantly improved image quality, issues like spectral distortions persist. To address this, we propose enhancements to the PSGAN framework by introducing novel regularization techniques for the generator loss function. Experimental results on images from the Worldview-3 dataset demonstrate that the proposed modifications improve spectral fidelity and achieve superior performance across multiple quantitative metrics while delivering visually superior results.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the Pansharpening problem in remote - sensing image fusion. Specifically, the paper aims to generate high - resolution multispectral images by fusing low - resolution multispectral images and high - resolution panchromatic images while maintaining or improving spectral fidelity. ### Specific Problems and Challenges: 1. **Spectral Distortion**: - Traditional methods often introduce spectral distortion when performing pansharpening, resulting in the spectral characteristics of the generated high - resolution multispectral images being inconsistent with those of the original multispectral images. 2. **Limitations of Spatial Detail Injection**: - Traditional methods have limitations in injecting spatial details, especially being unstable when dealing with different sensor data and ground object features. 3. **Limitations of Theoretical Assumptions**: - Some traditional methods rely on specific theoretical assumptions, which may not hold in practical applications, thus affecting the universality and performance of the methods. 4. **Improvement Requirements for Deep Learning Methods**: - Although existing deep learning methods have significantly improved image quality, the problem of spectral distortion still exists. Therefore, these methods need to be further improved to enhance spectral fidelity. ### Solutions Proposed in the Paper: To solve the above problems, the paper proposes improvements to the PSGAN framework, mainly by introducing new regularization techniques to optimize the generator loss function. Specific improvements include: - **SAM - based Loss Function**: Use Spectral Angular Mapper (SAM) as a regularization term to reduce spectral distortion. - **Dual - Resolution SAM Loss**: Apply SAM loss at down - sampled and original resolutions respectively, and use it as a regularization term in the generator loss function. - **Perceptual Loss**: Generate more accurate spectral images by minimizing the distance between the generated image and the original multispectral image in the high - level feature subspace. - **Gram - Matrix - based Perceptual Loss**: Calculate the Gram matrices of the high - level features of the generated image and the original image, and minimize the Euclidean distance between them. - **Gram - Matrix - based Reconstruction Loss**: Minimize the distance between the Gram matrices of the reference image and the pansharpened image as a regularization term in the generator loss function. Through these improvements, the paper aims to improve the quality of pansharpened images, especially to achieve better results in terms of spectral fidelity. Experimental results show that these improvements exhibit superior performance on multiple quantitative indicators and also have a significant improvement in visual effects.

Comprehensive Analysis and Improvements in Pansharpening Using Deep Learning

Pansharpening by Combining Enhanced Spatial-Spectral Fidelity and Gradient-Domain Guided Filtering

Pan-Sharpening Based On Parallel Pyramid Convolutional Neural Network

Towards Robust Pansharpening: A Large-Scale High-Resolution Multi-Scene Dataset and Novel Approach

PAN-Guided Band-Aware Multi-Spectral Feature Enhancement for Pan-Sharpening

A New Pansharpening Method Using Objectness Based Saliency Analysis and Saliency Guided Deep Residual Network

Zero-shot semi-supervised learning for pansharpening

A Multi-Stage Progressive Pansharpening Network Based on Detail Injection with Redundancy Reduction

Machine Learning in Pansharpening: A Benchmark, From Shallow to Deep Networks

An Improved Method for Pan-Sharpening Based on Pan-GAN

An Improved Version of the Generalized Laplacian Pyramid Algorithm for Pansharpening

Rethinking Pan-Sharpening Via Spectral-Band Modulation

Domain-specific knowledge-driven pan-sharpening algorithm

PSGAN: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening

An Unsupervised CNN-Based Pansharpening Framework with Spectral-Spatial Fidelity Balance

Memory-augmented Deep Conditional Unfolding Network for Pansharpening

Unsupervised Deep Learning-based Pansharpening with Jointly-Enhanced Spectral and Spatial Fidelity

Pan-GAN: An unsupervised learning method for pan-sharpening in remote sensing image fusion using a generative adversarial network

Deep Gradient Projection Networks for Pan-sharpening

PC-GANs: Progressive Compensation Generative Adversarial Networks for Pan-sharpening

Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion