Abstract:Structures matter in single image super-resolution (SISR). Benefiting from generative adversarial networks (GANs), recent studies have promoted the development of SISR by recovering photo-realistic images. However, there are still undesired structural distortions in the recovered images. In this paper, we propose a structure-preserving super-resolution (SPSR) method to alleviate the above issue while maintaining the merits of GAN-based methods to generate perceptual-pleasant details. First, we propose SPSR with gradient guidance (SPSR-G) by exploiting gradient maps of images to guide the recovery in two aspects. On the one hand, we restore high-resolution gradient maps by a gradient branch to provide additional structure priors for the SR process. On the other hand, we propose a gradient loss to impose a second-order restriction on the super-resolved images, which helps generative networks concentrate more on geometric structures. Second, since the gradient maps are handcrafted and may only be able to capture limited aspects of structural information, we further extend SPSR-G by introducing a learnable neural structure extractor (NSE) to unearth richer local structures and provide stronger supervision for SR. We propose two self-supervised structure learning methods, contrastive prediction and solving jigsaw puzzles, to train the NSEs. Our methods are model-agnostic, which can be potentially used for off-the-shelf SR networks. Experimental results on five benchmark datasets show that the proposed methods outperform state-of-the-art perceptual-driven SR methods under LPIPS, PSNR, and SSIM metrics. Visual results demonstrate the superiority of our methods in restoring structures while generating natural SR images. Code is available at https://github.com/Maclory/SPSR.

Learning Structral Coherence Via Generative Adversarial Network for Single Image Super-Resolution

Generative Adversarial Network-based Image Super-Resolution with a Novel Quality Loss

Structure-Preserving Super Resolution With Gradient Guidance

Single Remote Sensing Image Super-Resolution Via a Generative Adversarial Network with Stratified Dense Sampling and Chain Training

End-to-End Image Super-Resolution Via Generative Adversarial Network

Structure-Preserving Image Super-Resolution

Learning Spectral and Spatial Features Based on Generative Adversarial Network for Hyperspectral Image Super-Resolution

Hyperspectral Imagery Spatial Super-Resolution Using Generative Adversarial Network.

SRPGAN: Perceptual Generative Adversarial Network for Single Image Super Resolution.

Image Super-Resolution Reconstruction Using Generative Adversarial Networks Based on Wide-Channel Activation

DSRGAN: Detail Prior-Assisted Perceptual Single Image Super-Resolution via Generative Adversarial Networks

Structure-Aware Deep Networks and Pixel-Level Generative Adversarial Training for Single Image Super-Resolution.

Hierarchical Generative Adversarial Networks for Single Image Super-Resolution

Sharp And Real Image Super-Resolution Using Generative Adversarial Network

Generative Adversarial Network-Based Super-Resolution Considering Quantitative And Perceptual Quality

Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution

Gram-GAN: Image Super-Resolution Based on Gram Matrix and Discriminator Perceptual Loss

Image Super-Resolution Using a Improved Generative Adversarial Network

A Positive-Unlabeled Generative Adversarial Network for Super-Resolution Image Reconstruction Using a Charbonnier Loss

Single image super resolution based on generative adversarial networks

Super-Resolution of Remote Sensing Images via a Dense Residual Generative Adversarial Network