Abstract:To address the problems in the existing video super-resolution methods, such as noise, over smooth and visual artifacts, which are caused by the reliance on limited external training or mismatch of internal similarity patch instances, this study proposes a novel video super-resolution reconstruction algorithm based on deep learning and spatio-temporal feature similarity (DLSS-VSR). The video super-resolution reconstruction mechanism with the joint internal and external constraints is established utilizing the complementary advantages of both external deep correlation mapping learning and internal spatio-temporal nonlocal self-similarity prior constraint. A deep learning model based on deep convolutional neural network is constructed to learn the nonlinear correlation mapping between low-resolution and high-resolution video frame patches. A novel spatio-temporal feature similarity calculation method is proposed, which considers both internal video spatio-temporal self-similarity and external clean nonlocal similarity. For the internal spatio-temporal feature self-similarity, we improve the accuracy and robustness of similarity matching by proposing a similarity measure strategy based on spatio-temporal moment feature similarity and structural similarity. The external nonlocal similarity prior constraint is learned by the patch group-based Gaussian mixture model. The time efficiency for spatio-temporal similarity matching is further improved based on saliency detection and region correlation judgment strategy, which achieves a better tradeoff between super-resolution accuracy and speed. Experimental results demonstrate that the DLSS-VSR algorithm achieves competitive super-resolution quality compared to other state-of-the-art algorithms in both subjective and objective evaluations.

A Novel Super-Resolution Image and Video Reconstruction Approach Based on Newton-Thiele’s Rational Kernel in Sparse Principal Component Analysis

A Novel Two-Step Approach for the Super-Resolution Reconstruction of Video Sequences

Super-resolution by Polar Newton-Thiele's Rational Kernel in Centralized Sparsity Paradigm

Image Super-resolution Based on Tiny Recurrent Convolutional Neural Network

Super-resolution Reconstruction Based on Continued Fractions Interpolation Kernel in the Polar Coordinates

Practical super-resolution from dynamic video sequences

An Efficient Algorithm for Video Super-Resolution Based On a Sequential Model

A Sparse Representation Based Super-Resolution Image Reconstruction Scheme Utilizing Dual Dictionaries

A Fast Super-Resolution Method Based on Sparsity Properties.

Video Super-Resolution Reconstruction Based on Deep Learning and Spatio-Temporal Feature Self-similarity

Video Super-Resolution Via Pre-Frame Constrained and Deep-Feature Enhanced Sparse Reconstruction

Video Super-Resolution Algorithm Based on Spatial-Temporal Feature and Neural Network

A novel super resolution reconstruction of low reoslution images progressively using dct and zonal filter based denoising

Noise-robust Video Super-Resolution Using an Adaptive Spatial-Temporal Filter

Single Image Super-Resolution Using Sparse Prior

Kernel based low-rank sparse model for single image super-resolution

A noise-suppressing and edge-preserving multiframe super-resolution image reconstruction method

A Robust Video Super-Resolution Algorithm.

Adaptive algorithm of super-resolution image reconstruction

Single-Image Super-Resolution Via an Iterative Reproducing Kernel Hilbert Space Method.

Image super-resolution via sparse representation.