Abstract:Reference-based Super-Resolution (Ref-SR) has recently emerged as a promising paradigm to enhance a low-resolution (LR) input image or video by introducing an additional high-resolution (HR) reference image. Existing Ref-SR methods mostly rely on implicit correspondence matching to borrow HR textures from reference images to compensate for the information loss in input images. However, performing local transfer is difficult because of two gaps between input and reference images: the transformation gap (e.g., scale and rotation) and the resolution gap (e.g., HR and LR). To tackle these challenges, we propose C2-Matching in this work, which performs explicit robust matching crossing transformation and resolution. 1) To bridge the transformation gap, we propose a contrastive correspondence network, which learns transformation-robust correspondences using augmented views of the input image. 2) To address the resolution gap, we adopt teacher-student correlation distillation, which distills knowledge from the easier HR-HR matching to guide the more ambiguous LR-HR matching. 3) Finally, we design a dynamic aggregation module to address the potential misalignment issue between input images and reference images. In addition, to faithfully evaluate the performance of Reference-based Image Super-Resolution (Ref Image SR) under a realistic setting, we contribute the Webly-Referenced SR (WR-SR) dataset, mimicking the practical usage scenario. We also extend C2-Matching to Reference-based Video Super-Resolution (Ref VSR) task, where an image taken in a similar scene serves as the HR reference image. Extensive experiments demonstrate that our proposed C2-Matching significantly outperforms state of the arts by up to 0.7 dB on the standard CUFED5 benchmark and also boosts the performance of video super-resolution by incorporating the C2-Matching component into Video SR pipelines. Notably, C2-Matching also shows great generalizability on WR-SR dataset as well as robustness across large scale and rotation transformations. Codes and datasets are available at https://github.com/yumingj/C2-Matching.

Reduced-reference Quality Assessment of Image Super-Resolution by Energy Change and Texture Variation

Perception- and Fidelity-aware Reduced-Reference Super-Resolution Image Quality Assessment

Textural-Perceptual Joint Learning for No-Reference Super-Resolution Image Quality Assessment

QMRNet: Quality Metric Regression for EO Image Quality Assessment and Super-Resolution

Quality Assessment of Image Super-Resolution: Balancing Deterministic and Statistical Fidelity

Single Image Super-Resolution Quality Assessment: A Real-World Dataset, Subjective Studies, and an Objective Metric

SPQE: Structure-and-Perception-Based Quality Evaluation for Image Super-Resolution

No Reference Quality Assessment for Screen Content Images With Both Local and Global Feature Representation

A Deep Learning Based No-Reference Image Quality Assessment Model For Single-Image Super-Resolution

Reduced-Reference 3D Image Quality Measurement via Spatial to Gradient Domain Feature Aggregation

Reference-Based Super-Resolution Method for Remote Sensing Images with Feature Compression Module

Internal Generative Mechanism Inspired Reduced Reference Image Quality Assessment with Entropy of Primitive.

A Feature Reuse Framework with Texture-adaptive Aggregation for Reference-based Super-Resolution

Blind Quality Assessment for Image Superresolution Using Deep Two-Stream Convolutional Networks

A natural image quality evaluation metric

Deep Objective Quality Assessment Driven Single Image Super-Resolution

Reference-Based Image and Video Super-Resolution via C2-Matching

Detail-Enhancing Framework for Reference-Based Image Super-Resolution

Reduced-Reference Quality Assessment of Screen Content Images

No Reference Image Quality Assessment Using Sparse Feature Representation in Two Dimensions Spatial Correlation