Abstract:To address the problems in the existing video super-resolution methods, such as noise, over smooth and visual artifacts, which are caused by the reliance on limited external training or mismatch of internal similarity patch instances, this study proposes a novel video super-resolution reconstruction algorithm based on deep learning and spatio-temporal feature similarity (DLSS-VSR). The video super-resolution reconstruction mechanism with the joint internal and external constraints is established utilizing the complementary advantages of both external deep correlation mapping learning and internal spatio-temporal nonlocal self-similarity prior constraint. A deep learning model based on deep convolutional neural network is constructed to learn the nonlinear correlation mapping between low-resolution and high-resolution video frame patches. A novel spatio-temporal feature similarity calculation method is proposed, which considers both internal video spatio-temporal self-similarity and external clean nonlocal similarity. For the internal spatio-temporal feature self-similarity, we improve the accuracy and robustness of similarity matching by proposing a similarity measure strategy based on spatio-temporal moment feature similarity and structural similarity. The external nonlocal similarity prior constraint is learned by the patch group-based Gaussian mixture model. The time efficiency for spatio-temporal similarity matching is further improved based on saliency detection and region correlation judgment strategy, which achieves a better tradeoff between super-resolution accuracy and speed. Experimental results demonstrate that the DLSS-VSR algorithm achieves competitive super-resolution quality compared to other state-of-the-art algorithms in both subjective and objective evaluations.

Deep Parametric 3D Filters for Joint Video Denoising and Illumination Enhancement in Video Super Resolution.

Video super-resolution with phase-aided deformable alignment network

Deep Parameterized Neural Networks for Hyperspectral Image Denoising

Video Super-Resolution Via a Spatio-Temporal Alignment Network.

Unified Single-Image and Video Super-Resolution via Denoising Algorithms

Video Super-Resolution Reconstruction Based on Deep Learning and Spatio-Temporal Feature Self-similarity

Towards High-Quality and Efficient Video Super-Resolution via Spatial-Temporal Data Overfitting

Video super-resolution with 3D adaptive normalized convolution

Iterative Back Projection Network Based on Deformable 3D Convolution

Joint Non-Gaussian Denoising and Superresolving of Raw High Frame Rate Videos

Fast Spatio-Temporal Residual Network For Video Super-Resolution

Deformable 3D Convolution for Video Super-Resolution

Deformable Kernel Convolutional Network for Video Extreme Super-Resolution

Video Super-Resolution Reconstruction Based on Deep Convolutional Neural Network and Spatio-Temporal Similarity

Structure-preserving video super-resolution using three-dimensional convolutional neural networks

Joint Depth Map Super-Resolution Method via Deep Hybrid-Cross Guidance Filter

An Efficient Network Design for Face Video Super-resolution

Noise-robust Video Super-Resolution Using an Adaptive Spatial-Temporal Filter

Towards Real-World Video Denosing: A Practical Video Denosing Dataset and Network

A Dual-Network Based Super-Resolution for Compressed High Definition Video.

Super-Resolution for Noisy Images Via Deep Convolutional Neural Network