Abstract:Making full use of spatial-temporal information is the key factor for removing compressed video artifacts. Recently, many deep learning-based compression artifact reduction methods have emerged. Among them, a series of methods based on deformable convolution have shown excellent capabilities in spatio-temporal feature extraction. However, local deformable offset prediction and pixel-wise inter-frame feature alignment in the unidirectional form limit the full utilization of temporal features in the existing method. Additionally, compressed video shows inconsistent degrees of distortion on different frequency components, and their restoration difficulty is also nonuniform. For the above problems presented by existing methods, we propose an enlarged motion-aware and frequency-aware network (EMAFA) to further extract spatio-temporal information and enhance information of different frequency components. To perceive different degrees of motion artifacts between compressed frames as accurately as possible, we design a bidirectional dense propagation pattern with pixel-wise and patch-wise deformable convolution (PIPA) module in the feature domain. In addition, we propose a multi-scale atrous deformable alignment (MSADA) module to enrich spatio-temporal features in image domain. Moreover, we design a multi-direction frequency enhancement (MDFE) module with multiple direction convolution to enhance the features of different frequency components. The experimental results show that the proposed method performs better than the state-of-the-art methods in both objective evaluation and visual perception experience. Supplementary experiments for Internet Streamed Video with hybrid-distortion demonstrate that our method also exhibits considerable generalizability for quality enhancement.

Characterizing Perceptual Artifacts in Compressed Video Streams

Saliency-Aware Spatio-Temporal Artifact Detection for Compressed Video Quality Assessment

Perceived Coding Distortion Assessment for Streaming Video

Spatio-Temporal Deformable Convolution for Compressed Video Quality Enhancement

Coding Distortion Assessment for Streaming Video

Video Compression Artifact Reduction via Spatio-Temporal Multi-Hypothesis Prediction.

A New Framework Based on Spatio-Temporal Information for Enhancing Compressed Video

Perception of Temporal Pumping Artifact in Video Coding with the Hierarchical Prediction Structure

An Efficient Algorithm to Eliminate Temporal Pumping Artifact in Video Coding with Hierarchical Prediction Structure

Perception-based Quantitative Definition of Temporal Pumping Artifact

A Survey on Perceptually Optimized Video Coding

A Frame Level Metric for Just Noticeable Temporal Pumping Artifact in Videos Encoded with the Hierarchical Prediction Structure.

Artifact reduction of compressed video via three-dimensional adaptive estimation of transform coefficients

Reduction of temporal distortion in video coding based on detection of just-noticeable temporal pumping artifact.

Temporal Distortions in H.265/Hevc

Perceptual Quality Metric For Compressed Videos

PEA265: Perceptual Assessment of Video Compression Artifacts

Research on Application of Perceptual Model Based Image Compression

Perceptual video streaming by adaptive spatial-temporal scalability

Enlarged Motion-Aware and Frequency-Aware Network for Compressed Video Artifact Reduction

A Video Compression Artifact Reduction Approach Combined with Quantization Parameters Estimation