Abstract:For video super-resolution, current state-of-the-art approaches either process multiple low-resolution (LR) frames to produce each output high-resolution (HR) frame separately in a sliding window fashion or recurrently exploit the previously estimated HR frames to super-resolve the following frame. The main weaknesses of these approaches are: 1) separately generating each output frame may obtain high-quality HR estimates while resulting in unsatisfactory flickering artifacts, and 2) combining previously generated HR frames can produce temporally consistent results in the case of short information flow, but it will cause significant jitter and jagged artifacts because the previous super-resolving errors are constantly accumulated to the subsequent frames. In this paper, we propose a fully end-to-end trainable frame and feature-context video super-resolution (FFCVSR) network that consists of two key sub-networks: local network and context network, where the first one explicitly utilizes a sequence of consecutive LR frames to generate local feature and local SR frame, and the other combines the outputs of local network and the previously estimated HR frames and features to super-resolve the subsequent frame. Our approach takes full advantage of the inter-frame information from multiple LR frames and the context information from previously predicted HR frames, producing temporally consistent high-quality results while maintaining real-time speed by directly reusing previous features and frames. Extensive evaluations and comparisons demonstrate that our approach produces state-of-the-art results on a standard benchmark dataset, with advantages in terms of accuracy, efficiency, and visual quality over the existing approaches.

High-Resolution Projection Network Combining High-Resolution Optical Flow Compensation for Video Super-Resolution

Video Super-Resolution Based on Multiple Networks Merging

Video super-resolution with phase-aided deformable alignment network

Deep Video Super-Resolution Using HR Optical Flow Estimation.

Attention deep and shallow network union HR optical flow compensation for video super-resolution

Learning for Video Super-Resolution through HR Optical Flow Estimation

Multi-Stage Feature Fusion Network for Video Super-Resolution

Video Super-Resolution Via a Spatio-Temporal Alignment Network.

DVSRNet: Deep Video Super-Resolution Based on Progressive Deformable Alignment and Temporal-Sparse Enhancement

Video Super-Resolution Via Nonlocal Deformable Alignment and Frame Recursive Progressive Fusion Network

Deep Video Super-Resolution with Flow-Guided Deformable Alignment and Sparsity-based Temporal-Spatial Enhancement

Frame and Feature-Context Video Super-Resolution

FGBRSN: Flow-Guided Gated Bi-Directional Recurrent Separated Network for Video Super-Resolution.

Video Super-Resolution Network Via Enhanced Deep Feature Extraction and Residual Up-Down Block

Enhanced Bidirectional Propagation Network for Video Super-Resolution

Optical Flow Super-Resolution Based on Image Guidence Using Convolutional Neural Network

Frequency Domain Based Super-Resolution Method for Mixed-Resolution Multi-View Images

Optical Flow Reusing for High-Efficiency Space-Time Video Super Resolution

Multiframe Video Satellite Image Super-Resolution via Attention-Based Residual Learning

HSTR-Net: High Spatio-Temporal Resolution Video Generation For Wide Area Surveillance

FFFN: Frame-By-Frame Feedback Fusion Network for Video Super-Resolution