Abstract:Fractional interpolation has been extensively utilized in a series of video coding standards to generate fractional precision prediction to remove the temporal redundancy in consecutive frames. In the traditional interpolation filter based methods, the fractional samples are interpolated through a linear combination of the neighboring integer samples. This method is simple yet unable to accurately characterize the nonstationary video signals. Recently, convolutional neural network has been utilized in the fractional interpolation and shows superior performance compared with the traditional methods. However, only the reconstruction of the reference frame is used as the infer information source. All the other information contained in the bitstream or generated during the encoding/decoding procedure denoted as the compression prior is not utilized at all. In this paper, we give the first trial to involve some compression priors into the CNN to improve the performance of a in-loop coding tool. Specifically, we propose a Compression Priors assisted Convolutional Neural Network (CPCNN) to further improve the fractional interpolation efficiency. In addition to the reconstructed component, we additionally utilize two other compression priors – the corresponding residual component and col-located high quality component to boost the performance. Specifically, the residual component that indicates the prediction efficiency and contains effective texture information is utilized as a complementary input to the reconstructed one. While the col-located component provides more useful high quality information to help the reconstruction get rid of the quality fluctuation. Furthermore, a special network structure is designed to learn powerful representations of these triple input components. Comprehensive experiments have been conducted to demonstrate the effectiveness of our proposed CPCNN. The experimental results show that compared to HEVC, our proposed CPCNN ac-ieves on average of 5.3%, 2.8% and 1.9% BD-Rate savings under LDP, LDB and RA configurations, respectively.

Convolutional Neural Network-Based Invertible Half-Pixel Interpolation Filter for Video Coding.

Invertibility-Driven Interpolation Filter for Video Coding.

Compression Priors Assisted Convolutional Neural Network for Fractional Interpolation

Improved CNN-based Learning of Interpolation Filters for Low-Complexity Inter Prediction in Video Coding

One-for-All: Grouped Variation Network-Based Fractional Interpolation in Video Coding

A Group Variational Transformation Neural Network for Fractional Interpolation of Video Coding

An Integrated CNN-based Post Processing Filter For Intra Frame in Versatile Video Coding

Content-Aware Convolutional Neural Network for In-Loop Filtering in High Efficiency Video Coding

Convolutional Neural Network-Based Fractional-Pixel Motion Compensation

Convolutional Neural Network Based In-Loop Filter for VVC Intra Coding

Switch Mode Based Deep Fractional Interpolation in Video Coding

Convolutional Neural Network-Based Block Up-Sampling for Intra Frame Coding

Multi-Gradient Convolutional Neural Network Based In-Loop Filter For Vvc

Rate-Distortion Optimization-Based Learned Fractional Interpolation Filter Design for HEVC

Multi-Density Convolutional Neural Network for In-Loop Filter in Video Coding.

Spatio-Temporal Convolutional Neural Network for Enhanced Inter Prediction in Video Coding

A Learning-Based Low Complexity In-Loop Filter for Video Coding

A Low Complexity and High Performance Interpolation Filter for MPEG IVC

A CNN-Based In-Loop Filter with CU Classification for HEVC

Joint Rate Distortion Optimization with CNN-based In-Loop Filter for Hybrid Video Coding

A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding