Abstract:Fractional interpolation has been extensively utilized in a series of video coding standards to generate fractional precision prediction to remove the temporal redundancy in consecutive frames. In the traditional interpolation filter based methods, the fractional samples are interpolated through a linear combination of the neighboring integer samples. This method is simple yet unable to accurately characterize the nonstationary video signals. Recently, convolutional neural network has been utilized in the fractional interpolation and shows superior performance compared with the traditional methods. However, only the reconstruction of the reference frame is used as the infer information source. All the other information contained in the bitstream or generated during the encoding/decoding procedure denoted as the compression prior is not utilized at all. In this paper, we give the first trial to involve some compression priors into the CNN to improve the performance of a in-loop coding tool. Specifically, we propose a Compression Priors assisted Convolutional Neural Network (CPCNN) to further improve the fractional interpolation efficiency. In addition to the reconstructed component, we additionally utilize two other compression priors – the corresponding residual component and col-located high quality component to boost the performance. Specifically, the residual component that indicates the prediction efficiency and contains effective texture information is utilized as a complementary input to the reconstructed one. While the col-located component provides more useful high quality information to help the reconstruction get rid of the quality fluctuation. Furthermore, a special network structure is designed to learn powerful representations of these triple input components. Comprehensive experiments have been conducted to demonstrate the effectiveness of our proposed CPCNN. The experimental results show that compared to HEVC, our proposed CPCNN ac-ieves on average of 5.3%, 2.8% and 1.9% BD-Rate savings under LDP, LDB and RA configurations, respectively.

Convolutional Block Design for Learned Fractional Downsampling

Learned Fractional Downsampling Network for Adaptive Video Streaming

Convolutional Neural Network-Based Block Up-Sampling for Intra Frame Coding

Learning A Convolutional Neural Network For Fractional Interpolation In Hevc Inter Coding

Attention-Based Dual-Scale CNN In-Loop Filter for Versatile Video Coding.

Compression Priors Assisted Convolutional Neural Network for Fractional Interpolation

An Adaptive and Fast Fractional Pixel Search Algorithm in H.264

Improved Low-Bitrate HEVC Video Coding Using Deep Learning Based Super-Resolution and Adaptive Block Patching.

Volume Upscaling with Convolutional Neural Networks

Convolutional Neural Network-Based Invertible Half-Pixel Interpolation Filter for Video Coding.

A Convolutional Neural Network-Based Low Complexity Filter

One-for-All: Grouped Variation Network-Based Fractional Interpolation in Video Coding

Novel algorithm for fractional resizing of digital image in DCT domain

Advanced CNN Based Motion Compensation Fractional Interpolation

An Efficient Low-Complexity Convolutional Neural Network Filter

Adaptive Fractional Dilated Convolution Network for Image Aesthetics Assessment

Convolutional Neural Network-Based Fractional-Pixel Motion Compensation

A Learning-Based Low Complexity In-Loop Filter for Video Coding

RR-DnCNN v2.0: Enhanced Restoration-Reconstruction Deep Neural Network for Down-Sampling-Based Video Coding

Deep Convolutional Neural Network For Decompressed Video Enhancement

A Distortion-Aware Multi-Task Learning Framework for Fractional Interpolation in Video Coding.