Abstract:Large-factor image super-resolution (SR) is a challenging task due to the high uncertainty and incompleteness of the missing details to be recovered. In remote sensing images, the subpixel spectral mixing and semantic ambiguity of ground objects make this task even more challenging. In this article, we propose a novel method for large-factor SR of remote sensing images named spectra-guided generative adversarial networks (SpecGANs). In response to the above problems, we explore whether introducing additional hyperspectral images (HSIs) to GAN as conditional input can be the key to solving the problems. Different from previous approaches that mainly focus on improving the feature representation of a single source input, we propose a dual-branch network architecture to effectively fuse low-resolution (LR) red, green, blue (RGB) images and corresponding HSIs, which fully exploit the rich hyperspectral information as conditional semantic guidance. Due to the spectral specificity of ground objects, the semantic accuracy of the generated images is guaranteed. To further improve the visual fidelity of the generated output, we also introduce the Latent Code Bank with rich visual priors under a generative adversarial training framework so that high-resolution, detailed, and realistic images can be progressively generated. Extensive experiments show the superiority of our method over the state-of-art image SR methods in terms of both quantitative evaluation metrics and visual quality. Ablation experiments also suggest the necessity of adding spectral information and the effectiveness of our designed fusion module. To our best knowledge, we are the first to achieve up to 32x SR of remote sensing images with high visual fidelity under the premise of accurate ground object semantics. Our code can be publicly available at https://github.com/YapengMeng/SpecGAN.

Multitemporal Hyperspectral Image Super-Resolution Through 3D Generative Adversarial Network

Super-Resolution for Hyperspectral Remote Sensing Images Based on the 3D Attention-SRGAN Network

Infrared Image Super Resolution Using Gan With Infrared Image Prior

Single Remote Sensing Image Super-Resolution Via a Generative Adversarial Network with Stratified Dense Sampling and Chain Training

Multiattention Generative Adversarial Network for Remote Sensing Image Super-Resolution

A Latent Encoder Coupled Generative Adversarial Network (LE-GAN) for Efficient Hyperspectral Image Super-resolution

Large-Factor Super-Resolution of Remote Sensing Images With Spectra-Guided Generative Adversarial Networks

VARIABLE SELECTION IN ROBUST JOINT MEAN AND COVARIANCE MODEL FOR LONGITUDINAL DATA ANALYSIS

Super-Resolution of Remote Sensing Images Based on Transferred Generative Adversarial Network

E-DBPN: Enhanced Deep Back-Projection Networks for Remote Sensing Scene Image Superresolution

Progressive split-merge super resolution for hyperspectral imagery with group attention and gradient guidance

Hyperspectral Image Super-Resolution via Dual-domain Network Based on Hybrid Convolution

3DAttGAN: A 3D Attention-based Generative Adversarial Network for Joint Space-Time Video Super-Resolution

Spectral–Spatial Generative Adversarial Network for Super-Resolution Land Cover Mapping With Multispectral Remotely Sensed Imagery

Hyperspectral Image Super-Resolution via Deep Spatiospectral Attention Convolutional Neural Networks

Hyperspectral Image Super-resolution via Deep Spatio-spectral Convolutional Neural Networks

Remote Sensing Image Super-Resolution Using Texture Enhancing Generative Adversarial Network

Remote Sensing Hyperspectral Image Super-Resolution via Multidomain Spatial Information and Multiscale Spectral Information Fusion

Mars Image Super-Resolution Based on Generative Adversarial Network

A Super-Resolution Algorithm Based on Hybrid Network for Multi-Channel Remote Sensing Images

Superresolution reconstruction of optical remote sensing images based on a multiscale attention adversarial network