Abstract:In this paper, we consider two challenging issues in reference-based super-resolution (RefSR) for smartphone, (i) how to choose a proper reference image, and (ii) how to learn RefSR in a self-supervised manner. Particularly, we propose a novel self-supervised learning approach for real-world RefSR from observations at dual and multiple camera zooms. Firstly, considering the popularity of multiple cameras in modern smartphones, the more zoomed (telephoto) image can be naturally leveraged as the reference to guide the super-resolution (SR) of the lesser zoomed (ultra-wide) image, which gives us a chance to learn a deep network that performs SR from the dual zoomed observations (DZSR). Secondly, for self-supervised learning of DZSR, we take the telephoto image instead of an additional high-resolution image as the supervision information, and select a center patch from it as the reference to super-resolve the corresponding ultra-wide image patch. To mitigate the effect of the misalignment between ultra-wide low-resolution (LR) patch and telephoto ground-truth (GT) image during training, we first adopt patch-based optical flow alignment and then design an auxiliary-LR to guide the deforming of the warped LR features. To generate visually pleasing results, we present local overlapped sliced Wasserstein loss to better represent the perceptual difference between GT and output in the feature space. During testing, DZSR can be directly deployed to super-solve the whole ultra-wide image with the reference of the telephoto image. In addition, we further take multiple zoomed observations to explore self-supervised RefSR, and present a progressive fusion scheme for the effective utilization of reference images. Experiments show that our methods achieve better quantitative and qualitative performance against state-of-the-arts. Codes are available at

Wisdom of Crowds: Single Image Super-resolution from the Web

Super-resolution Reconstruction Algorithms Based on Fusion of Deep Learning Mechanism and Wavelet

Epistemic-Uncertainty-Based Divide-and-Conquer Network for Single-Image Super-Resolution

Single Remote Sensing Image Super-Resolution Via a Generative Adversarial Network with Stratified Dense Sampling and Chain Training

Deep Learning Based Single Image Super-resolution: A Survey

Super-resolution for Low Quality Thumbnail Images

Exploiting Raw Images for Real-Scene Super-Resolution

Self-supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations.

Robust web image/video super-resolution

Self-Supervised Learning for Real-World Super-Resolution from Dual and Multiple Zoomed Observations

Landmark Image Super-Resolution by Retrieving Web Images

Super-Resolution Information Enhancement For Crowd Counting

A Remote Sensing Image Super-Resolution Reconstruction Model Combining Multiple Attention Mechanisms

Attention-based Multi-Reference Learning for Image Super-Resolution

Deep Learning for Single Image Super-Resolution: A Brief Review

Deep Learning for Image Super-Resolution: A Survey

Learning from Multi-Perception Features for Real-Word Image Super-resolution

Direct Unsupervised Super-Resolution Using Generative Adversarial Network (DUS-GAN) for Real-World Data

Single-Image Superresolution Based on Local Regression and Nonlocal Self-Similarity

SINGLE IMAGE SUPER-RESOLUTION USING A NON-LOCAL 3D CONVOLUTIONAL NEURAL NETWORK

Single Image Super-Resolution Methods: A Survey