Abstract:Large-factor image super-resolution (SR) is a challenging task due to the high uncertainty and incompleteness of the missing details to be recovered. In remote sensing images, the subpixel spectral mixing and semantic ambiguity of ground objects make this task even more challenging. In this article, we propose a novel method for large-factor SR of remote sensing images named spectra-guided generative adversarial networks (SpecGANs). In response to the above problems, we explore whether introducing additional hyperspectral images (HSIs) to GAN as conditional input can be the key to solving the problems. Different from previous approaches that mainly focus on improving the feature representation of a single source input, we propose a dual-branch network architecture to effectively fuse low-resolution (LR) red, green, blue (RGB) images and corresponding HSIs, which fully exploit the rich hyperspectral information as conditional semantic guidance. Due to the spectral specificity of ground objects, the semantic accuracy of the generated images is guaranteed. To further improve the visual fidelity of the generated output, we also introduce the Latent Code Bank with rich visual priors under a generative adversarial training framework so that high-resolution, detailed, and realistic images can be progressively generated. Extensive experiments show the superiority of our method over the state-of-art image SR methods in terms of both quantitative evaluation metrics and visual quality. Ablation experiments also suggest the necessity of adding spectral information and the effectiveness of our designed fusion module. To our best knowledge, we are the first to achieve up to 32x SR of remote sensing images with high visual fidelity under the premise of accurate ground object semantics. Our code can be publicly available at https://github.com/YapengMeng/SpecGAN.

Large-Factor Super-Resolution of Remote Sensing Images With Spectra-Guided Generative Adversarial Networks

Single Remote Sensing Image Super-Resolution Via a Generative Adversarial Network with Stratified Dense Sampling and Chain Training

Hyperspectral Imagery Spatial Super-Resolution Using Generative Adversarial Network.

Learning Spectral and Spatial Features Based on Generative Adversarial Network for Hyperspectral Image Super-Resolution

Multitemporal Hyperspectral Image Super-Resolution Through 3D Generative Adversarial Network

Hyperspectral Image Super-Resolution by Band Attention Through Adversarial Learning.

E-DBPN: Enhanced Deep Back-Projection Networks for Remote Sensing Scene Image Superresolution

Spectral–Spatial Generative Adversarial Network for Super-Resolution Land Cover Mapping With Multispectral Remotely Sensed Imagery

Adversarial Spectral Super-Resolution for Multispectral Imagery Using Spatial Spectral Feature Attention Module

Multi-Attention Generative Adversarial Network for Remote Sensing Image Super-Resolution

Super-Resolution for Hyperspectral Remote Sensing Images Based on the 3D Attention-SRGAN Network

MSAGAN: A New Super-Resolution Algorithm for Multispectral Remote Sensing Image Based on a Multiscale Attention GAN Network

Multiattention Generative Adversarial Network for Remote Sensing Image Super-Resolution

Multi-Attention Generative Adversarial Network for Remote Sensing Image Super-Resolution

Hyperspectral Image Super-Resolution Using Generative Adversarial Network And Residual Learning

Generative Adversarial Networks for Hyperspectral Image Spatial Super-Resolution

A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution

Super-Resolution of Remote Sensing Images Based on Transferred Generative Adversarial Network

Super Resolution Generative Adversarial Network Based Image Augmentation for Scene Classification of Remote Sensing Images

An Improved Generative Adversarial Network for Remote Sensing Image Super-Resolution

Remote-sensing Image Super-Resolution Using Classifier-Based Generative Adversarial Networks