Abstract:In this paper, we consider two challenging issues in reference-based super-resolution (RefSR) for smartphone, (i) how to choose a proper reference image, and (ii) how to learn RefSR in a self-supervised manner. Particularly, we propose a novel self-supervised learning approach for real-world RefSR from observations at dual and multiple camera zooms. Firstly, considering the popularity of multiple cameras in modern smartphones, the more zoomed (telephoto) image can be naturally leveraged as the reference to guide the super-resolution (SR) of the lesser zoomed (ultra-wide) image, which gives us a chance to learn a deep network that performs SR from the dual zoomed observations (DZSR). Secondly, for self-supervised learning of DZSR, we take the telephoto image instead of an additional high-resolution image as the supervision information, and select a center patch from it as the reference to super-resolve the corresponding ultra-wide image patch. To mitigate the effect of the misalignment between ultra-wide low-resolution (LR) patch and telephoto ground-truth (GT) image during training, we first adopt patch-based optical flow alignment and then design an auxiliary-LR to guide the deforming of the warped LR features. To generate visually pleasing results, we present local overlapped sliced Wasserstein loss to better represent the perceptual difference between GT and output in the feature space. During testing, DZSR can be directly deployed to super-solve the whole ultra-wide image with the reference of the telephoto image. In addition, we further take multiple zoomed observations to explore self-supervised RefSR, and present a progressive fusion scheme for the effective utilization of reference images. Experiments show that our methods achieve better quantitative and qualitative performance against state-of-the-arts. Codes are available at

Continuous Digital Zoom with Cross Attention for Dual Camera System

Fast Image Super-Resolution for a Dual-Resolution Camera

Photo-realistic continuous digital zooming for an asymmetrical dual camera system

Continuous Digital Zooming Using Generative Adversarial Networks for Dual Camera System

Research on Zoom Algorithm Based on Dual Resolution Camera

Robust Dual Images Super-Resolution.

Dual-Focal Camera Hdr Imaging Based On Convolutional Neural Network

Superpixel-Based Linear Reconstruction Method for Dual-Camera Compressed Hyperspectral Imaging System

Self-supervised Learning for Real-World Super-Resolution from Dual Zoomed Observations.

Image super-resolution using CNNs for dual-focal camera system

Self-Supervised Learning for Real-World Super-Resolution from Dual and Multiple Zoomed Observations

DDet: Dual-path Dynamic Enhancement Network for Real-World Image Super-Resolution

CrossZoom: Simultaneous Motion Deblurring and Event Super-Resolving

Low-Light Color Imaging Via Cross-Camera Synthesis.

Dual-Camera Smooth Zoom on Mobile Phones

Camera Lens Super-Resolution

Cross-MPI: Cross-scale Stereo for Image Super-Resolution using Multiplane Images

DSMA: Reference-Based Image Super-Resolution Method Based on Dual-View Supervised Learning and Multi-Attention Mechanism

CrossZoom: Simultaneously Motion Deblurring and Event Super-Resolving

Low-Light Color Imaging via Dual Camera Acquisition.

Multi-Scale Spatial Transformation and Attention Based Super-Resolution Reconstruction Algorithm