Abstract:To address the problems in the existing video super-resolution methods, such as noise, over smooth and visual artifacts, which are caused by the reliance on limited external training or mismatch of internal similarity patch instances, this study proposes a novel video super-resolution reconstruction algorithm based on deep learning and spatio-temporal feature similarity (DLSS-VSR). The video super-resolution reconstruction mechanism with the joint internal and external constraints is established utilizing the complementary advantages of both external deep correlation mapping learning and internal spatio-temporal nonlocal self-similarity prior constraint. A deep learning model based on deep convolutional neural network is constructed to learn the nonlinear correlation mapping between low-resolution and high-resolution video frame patches. A novel spatio-temporal feature similarity calculation method is proposed, which considers both internal video spatio-temporal self-similarity and external clean nonlocal similarity. For the internal spatio-temporal feature self-similarity, we improve the accuracy and robustness of similarity matching by proposing a similarity measure strategy based on spatio-temporal moment feature similarity and structural similarity. The external nonlocal similarity prior constraint is learned by the patch group-based Gaussian mixture model. The time efficiency for spatio-temporal similarity matching is further improved based on saliency detection and region correlation judgment strategy, which achieves a better tradeoff between super-resolution accuracy and speed. Experimental results demonstrate that the DLSS-VSR algorithm achieves competitive super-resolution quality compared to other state-of-the-art algorithms in both subjective and objective evaluations.

Super-resolution Reconstruction Based on Multisource Bidirectional Similarity and Non-Local Similarity Matching

Learning-Based Video Superresolution Reconstruction Using Spatiotemporal Nonlocal Similarity

Single Image Super-Resolution with Multiscale Similarity Learning.

Single Image Super-Resolution Using Local Geometric Duality and Non-Local Similarity

A Sparse Representation Based Super-Resolution Image Reconstruction Scheme Utilizing Dual Dictionaries

A Single-Image Super-Resolution Method Based on Progressive-Iterative Approximation

Learning Local Dictionaries and Similarity Structures for Single Image Super-Resolution.

Spatio-Temporal Adaptive Super-Resolution Reconstruction Model Based on Zernike Moment for Spatial Video Sequences

Video Super-Resolution Reconstruction Based on Deep Convolutional Neural Network and Spatio-Temporal Similarity

Video Super-Resolution Reconstruction Based on Deep Learning and Spatio-Temporal Feature Self-similarity (extended Abstract)

Single Image Super-Resolution Based on Nonlocal Similarity and Sparse Representation

Video Super-Resolution Reconstruction Based on Deep Learning and Spatio-Temporal Feature Self-similarity

Illumination-Invariance And Nonlocal Means Based Super Resolution

Single Image Super-Resolution Via Locally Regularized Anchored Neighborhood Regression and Nonlocal Means.

Single-Image Superresolution Based on Local Regression and Nonlocal Self-Similarity

Spatio-temporal Super-Resolution Reconstruction Based on Robust Optical Flow and Zernike Moment for Dynamic Image Sequences.

Single Image Super Resolution Based on Multiscale Local Similarity and Neighbor Embedding.

Single-image Super-resolution Algorithm Based on Multi-scale Nonlocal Regularization

An Adaptive Method for Simultaneous Image Super-Resolution and Motion Estimation Based on Nonlinear Least Square

Single-image Super-Resolution Reconstruction Via Double Layer Reconstructing

A multi-frame adaptive super-resolution method using double channel and regional pixel information