Abstract:Video super-resolution (VSR), which takes advantage of multiple low-resolution (LR) video frames to reconstruct corresponding high-resolution (HR) frames in a video, has raised increasing interest. To upsample an LR frame (denoted by a reference frame), VSR methods usually align multiple neighboring frames (denoted by supporting frames) to the reference frame first in order to provide more relevant information. The existing VSR methods usually employ deformable convolution to conduct the frame alignment, where the whole supporting frame is aligned to the reference frame without a specific target and without supervision. Thus, the aligned features are not explicitly learned to provide the HR frame information and cannot fully explore the supporting frames. To address this problem, in this work, we propose a novel video super-resolution framework with Position-Guided Multi-Head Alignment, termed as PGMH-A, to explicitly align the supporting frames to different spatial positions of the HR frame (denoted by different heads). It injects explicit position information to obtain multi-head-aligned features of supporting frames to better formulate the HR frame. PGMH-A can be trained individually or end-to-end with the ground-truth HR frames. Moreover, a Position-Guided Multi-Head Fusion, termed as PGMH-F, is developed based on the attention mechanism to further fuse the spatial–temporal information across temporal supporting frames, across multiple heads corresponding to the different spatial positions of an HR frame, and across multiple channels. Together, the proposed Position-Guided Multi-Head Alignment and Fusion (PGMH-AF) can provide VSR with better local details and temporal coherence. The experimental results demonstrate that the proposed method outperforms the state-of-the-art VSR networks. Ablation studies have also been conducted to verify the effectiveness of the proposed modules.

Temporal-spatial feature compensation combines with multi-feature discriminators for video super-resolution perceptual quality improvement

Video super-resolution with phase-aided deformable alignment network

Single Remote Sensing Image Super-Resolution Via a Generative Adversarial Network with Stratified Dense Sampling and Chain Training

Video Super-Resolution Via a Spatio-Temporal Alignment Network.

Multi-Stage Feature Fusion Network for Video Super-Resolution

FM-VSR: Feature Multiplexing Video Super-Resolution for Compressed Video

Position-Guided Multi-Head Alignment and Fusion for Video Super-Resolution

VideoGigaGAN: Towards Detail-rich Video Super-Resolution

A multiresolution mixture generative adversarial network for video super-resolution

Generative Adversarial Networks and Perceptual Losses for Video Super-Resolution

Collaborative Feedback Discriminative Propagation for Video Super-Resolution

3DAttGAN: A 3D Attention-based Generative Adversarial Network for Joint Space-Time Video Super-Resolution

Video super-resolution via mixed spatial-temporal convolution and selective fusion

FRAGAN-VSR - Frame-Recurrent Attention Generative Adversarial Network for Video Super-Resolution.

Multi-Frame Super-Resolution Algorithm Based on a WGAN

DSRGAN: Detail Prior-Assisted Perceptual Single Image Super-Resolution via Generative Adversarial Networks

Frame and Feature-Context Video Super-Resolution

GCFSR: a Generative and Controllable Face Super Resolution Method Without Facial and GAN Priors

Gram-GAN: Image Super-Resolution Based on Gram Matrix and Discriminator Perceptual Loss

Retinal fundus image superresolution generated by optical coherence tomography based on a realistic mixed attention GAN

High-order relational generative adversarial network for video super-resolution