Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion

Jiayi Ma,Wei Yu,Chen Chen,Pengwei Liang,Xiaojie Guo,Junjun Jiang

DOI: https://doi.org/10.1016/j.inffus.2020.04.006

IF: 18.6

2020-10-01

Information Fusion

Abstract:<p>Pan-sharpening in remote sensing image fusion refers to obtaining multi-spectral images of high-resolution by fusing panchromatic images and multi-spectral images of low-resolution. Recently, convolution neural network (CNN)-based pan-sharpening methods have achieved the state-of-the-art performance. Even though, two problems still remain. On the one hand, the existing CNN-based strategies require supervision, where the low-resolution multi-spectral image is obtained by simply blurring and down-sampling the high-resolution one. On the other hand, they typically ignore rich spatial information of panchromatic images. To address these issues, we propose a novel unsupervised framework for pan-sharpening based on a generative adversarial network, termed as Pan-GAN, which does not rely on the so-called ground-truth during network training. In our method, the generator separately establishes the adversarial games with the spectral discriminator and the spatial discriminator, so as to preserve the rich spectral information of multi-spectral images and the spatial information of panchromatic images. Extensive experiments are conducted to demonstrate the effectiveness of the proposed Pan-GAN compared with other state-of-the-art pan-sharpening approaches. Our Pan-GAN has shown promising performance in terms of qualitative visual effects and quantitative evaluation metrics.</p>

computer science, artificial intelligence, theory & methods

What problem does this paper attempt to address?

The problem that this paper attempts to solve is in the Pan - sharpening task in remote sensing image fusion, how to simultaneously preserve the spectral information of the multispectral image and the spatial information of the panchromatic image without relying on the so - called "ground - truth". Specifically, the existing pan - sharpening methods based on convolutional neural networks (CNN) have two main problems:

1. **Supervision Requirement**: Existing methods usually require supervision, that is, obtaining low - resolution multispectral images by simply blurring and down - sampling high - resolution multispectral images, which may be unrealistic in actual scenarios because the relationship between high - and low - resolution images does not always follow simple blurring and down - sampling operations.
2. **Spatial Information Neglect**: These methods usually neglect the rich spatial information in the panchromatic image and only focus on the preservation of spectral information.

To solve these problems, the author proposes an unsupervised pan - sharpening framework based on the generative adversarial network (GAN), called Pan - GAN. This framework introduces two discriminators (one spectral discriminator and one spatial discriminator) to respectively force the generated image to be consistent with the low - resolution multispectral image and the panchromatic image in terms of spectral information and spatial information, thereby achieving high - quality pan - sharpening results.