Abstract:Video super-resolution (VSR), which takes advantage of multiple low-resolution (LR) video frames to reconstruct corresponding high-resolution (HR) frames in a video, has raised increasing interest. To upsample an LR frame (denoted by a reference frame), VSR methods usually align multiple neighboring frames (denoted by supporting frames) to the reference frame first in order to provide more relevant information. The existing VSR methods usually employ deformable convolution to conduct the frame alignment, where the whole supporting frame is aligned to the reference frame without a specific target and without supervision. Thus, the aligned features are not explicitly learned to provide the HR frame information and cannot fully explore the supporting frames. To address this problem, in this work, we propose a novel video super-resolution framework with Position-Guided Multi-Head Alignment, termed as PGMH-A, to explicitly align the supporting frames to different spatial positions of the HR frame (denoted by different heads). It injects explicit position information to obtain multi-head-aligned features of supporting frames to better formulate the HR frame. PGMH-A can be trained individually or end-to-end with the ground-truth HR frames. Moreover, a Position-Guided Multi-Head Fusion, termed as PGMH-F, is developed based on the attention mechanism to further fuse the spatial–temporal information across temporal supporting frames, across multiple heads corresponding to the different spatial positions of an HR frame, and across multiple channels. Together, the proposed Position-Guided Multi-Head Alignment and Fusion (PGMH-AF) can provide VSR with better local details and temporal coherence. The experimental results demonstrate that the proposed method outperforms the state-of-the-art VSR networks. Ablation studies have also been conducted to verify the effectiveness of the proposed modules.

Video Super-Resolution Using Multiple Complementary Priors

Multi-frame Image Super Resolution with Natural Image Prior

Video Super-Resolution Based on Multiple Networks Merging

Robust Super Resolution of Compressed Video

Video super-resolution with 3D adaptive normalized convolution

Satellite Video Super-Resolution Based on Adaptively Spatiotemporal Neighbors and Nonlocal Similarity Regularization

Single Image Super-Resolution with Non-Local Means and Steering Kernel Regression

Unified Single-Image and Video Super-Resolution via Denoising Algorithms

Practical super-resolution from dynamic video sequences

Video Super-Resolution Via a Spatio-Temporal Alignment Network.

Noise-robust Video Super-Resolution Using an Adaptive Spatial-Temporal Filter

Iterative Back Projection Network Based on Deformable 3D Convolution

Joint Non-Gaussian Denoising and Superresolving of Raw High Frame Rate Videos

Video Super-Resolution Reconstruction Based on Deep Learning and Spatio-Temporal Feature Self-similarity

Multiframe Video Satellite Image Super-Resolution via Attention-Based Residual Learning

An Efficient Algorithm for Video Super-Resolution Based On a Sequential Model

Video Super-Resolution Using Plug-and-Play Priors

Towards Interpretable Video Super-Resolution Via Alternating Optimization.

Video super-resolution via mixed spatial-temporal convolution and selective fusion

Position-Guided Multi-Head Alignment and Fusion for Video Super-Resolution

Deep Parametric 3D Filters for Joint Video Denoising and Illumination Enhancement in Video Super Resolution.