Abstract:Reference-based Super-Resolution (Ref-SR) has recently emerged as a promising paradigm to enhance a low-resolution (LR) input image or video by introducing an additional high-resolution (HR) reference image. Existing Ref-SR methods mostly rely on implicit correspondence matching to borrow HR textures from reference images to compensate for the information loss in input images. However, performing local transfer is difficult because of two gaps between input and reference images: the transformation gap (e.g., scale and rotation) and the resolution gap (e.g., HR and LR). To tackle these challenges, we propose C2-Matching in this work, which performs explicit robust matching crossing transformation and resolution. 1) To bridge the transformation gap, we propose a contrastive correspondence network, which learns transformation-robust correspondences using augmented views of the input image. 2) To address the resolution gap, we adopt teacher-student correlation distillation, which distills knowledge from the easier HR-HR matching to guide the more ambiguous LR-HR matching. 3) Finally, we design a dynamic aggregation module to address the potential misalignment issue between input images and reference images. In addition, to faithfully evaluate the performance of Reference-based Image Super-Resolution (Ref Image SR) under a realistic setting, we contribute the Webly-Referenced SR (WR-SR) dataset, mimicking the practical usage scenario. We also extend C2-Matching to Reference-based Video Super-Resolution (Ref VSR) task, where an image taken in a similar scene serves as the HR reference image. Extensive experiments demonstrate that our proposed C2-Matching significantly outperforms state of the arts by up to 0.7 dB on the standard CUFED5 benchmark and also boosts the performance of video super-resolution by incorporating the C2-Matching component into Video SR pipelines. Notably, C2-Matching also shows great generalizability on WR-SR dataset as well as robustness across large scale and rotation transformations. Codes and datasets are available at https://github.com/yumingj/C2-Matching.

Reference-Guided Deep Super-Resolution Via Manifold Localized External Compensation

Data-Driven External Compensation Guided Deep Networks for Image Super-Resolution

Dual Recovery Network with Online Compensation for Image Super-Resolution.

Face Super-Resolution Via Bilayer Contextual Representation

Face Super-Resolution Via Multilayer Locality-Constrained Iterative Neighbor Embedding and Intermediate Dictionary Learning.

Learning Discrete Representations From Reference Images for Large Scale Factor Image Super-Resolution

Manifold-preserving Single-Image Super-Resolution Based on Collaborative Representation Support

Ensemble Super-Resolution with a Reference Dataset

Single Image Super-Resolution Via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means.

Double-Layer Search and Adaptive Pooling Fusion for Reference-Based Image Super-Resolution

Coarse-to-Fine Embedded PatchMatch and Multi-Scale Dynamic Aggregation for Reference-Based Super-resolution

Local-And Holistic-Structure Preserving Image Super Resolution Via Deep Joint Component Learning

MASA-SR: Matching Acceleration and Spatial Adaptation for Reference-Based Image Super-Resolution

Reference-Based Image and Video Super-Resolution via C2-Matching

Reference-based image super-resolution with attention extraction and pooling of residuals

SRLSP: A Face Image Super-Resolution Algorithm Using Smooth Regression with Local Structure Prior

Dense Contrastive Learning and Depth Dynamic Aggregation for Reference-based Super-Resolution.

A multi-frame adaptive super-resolution method using double channel and regional pixel information

Deep Video Super-Resolution Using HR Optical Flow Estimation.

Single image super-resolution via learned representative features and sparse manifold embedding

Single Image Super-Resolution with Multiscale Similarity Learning.