Abstract:Structures matter in single image super-resolution (SISR). Benefiting from generative adversarial networks (GANs), recent studies have promoted the development of SISR by recovering photo-realistic images. However, there are still undesired structural distortions in the recovered images. In this paper, we propose a structure-preserving super-resolution (SPSR) method to alleviate the above issue while maintaining the merits of GAN-based methods to generate perceptual-pleasant details. First, we propose SPSR with gradient guidance (SPSR-G) by exploiting gradient maps of images to guide the recovery in two aspects. On the one hand, we restore high-resolution gradient maps by a gradient branch to provide additional structure priors for the SR process. On the other hand, we propose a gradient loss to impose a second-order restriction on the super-resolved images, which helps generative networks concentrate more on geometric structures. Second, since the gradient maps are handcrafted and may only be able to capture limited aspects of structural information, we further extend SPSR-G by introducing a learnable neural structure extractor (NSE) to unearth richer local structures and provide stronger supervision for SR. We propose two self-supervised structure learning methods, contrastive prediction and solving jigsaw puzzles, to train the NSEs. Our methods are model-agnostic, which can be potentially used for off-the-shelf SR networks. Experimental results on five benchmark datasets show that the proposed methods outperform state-of-the-art perceptual-driven SR methods under LPIPS, PSNR, and SSIM metrics. Visual results demonstrate the superiority of our methods in restoring structures while generating natural SR images. Code is available at https://github.com/Maclory/SPSR.

Exploiting multi-scale spatial structures for sparsity based single image super-resolution

Image Super-Resolution Based on Structure-Modulated Sparse Representation.

A Sparse Representation Based Super-Resolution Image Reconstruction Scheme Utilizing Dual Dictionaries

Image Super-Resolution Via Nonlocal Similarity and Group Structured Sparse Representation

Single image super-resolution using feature adaptive learning and global structure sparsity

Single Image Superresolution Via Directional Group Sparsity and Directional Features.

Single-image Super-resolution Algorithm Based on Multi-scale Nonlocal Regularization

Single Image Super Resolution Based on Multi-scale Structural Self-similarity

Image super-resolution via sparse representation.

Single Image Super-Resolution Based on Nonlocal Sparse and Low-Rank Regularization.

Structure-Preserving Image Super-Resolution

A Single-Image Super-Resolution Method Via Low-Rank Matrix Recovery And Nonlinear Mappings

Single Image Super-Resolution Based on Space Structure Learning

Retrieval Compensated Group Structured Sparsity for Image Super-Resolution

Image Super-Resolution by Structural Sparse Coding

Single Image Super-Resolution Using Sparse Prior

Single Image Super-Resolution Based on Nonlocal Similarity and Sparse Representation

Hardware-friendly Scalable Image Super Resolution with Progressively Structured Sparsity

Single Image Super-Resolution with Multiscale Similarity Learning.

Multilevel Feature Exploration Network for Image Superresolution

Structure-Preserving Super Resolution With Gradient Guidance