Abstract:We propose a convolutional neural network-based coefficients prediction (CNNCP) method for intra-predicted residues in the High Efficiency Video Coding (HEVC) standard. In HEVC, discrete cosine transform (DCT) or discrete sine transform (DST) is adopted to convert the intra-predicted residues in the spatial domain into coefficients in the frequency domain. Each coefficient is scalar quantized and entropy coded into the bitstream. As DCT or DST is non-optimal linear transform, there still exist linear and non-linear correlations among different coefficients after the transform. In addition, there exist coefficients' correlations between current block and neighboring blocks, as these correlations cannot be completely exploited in the intra prediction. We thus propose to perform coefficients prediction to further reduce the redundancy among coefficients. The coefficients prediction is achieved using trained convolutional neural networks (CNNs), as CNNs can build complex relationship between input and output by training with a lot of data. In addition, a flag that signals whether to perform coefficients prediction or not at the coding unit level is transmitted to decoder. The proposed CNNCP method is implemented upon the HEVC reference software. Experimental results show that the proposed method achieves on average 1.8%, 4.1%, and 4.5% BD-rate reduction ratios in Y, U, V, respectively, compared with the HEVC baseline in all-intra configuration. In particular, the average BD-rate reduction ratios for 4K test sequences are 2.9%, 6.5%, and 6.6%.

CNN-Based Bi-Directional Motion Compensation for High Efficiency Video Coding.

Convolutional Neural Network-Based Motion Compensation Refinement For Video Coding

Enhanced Bi-Prediction with Convolutional Neural Network for High-Efficiency Video Coding

Enhanced Ctu-Level Inter Prediction with Deep Frame Rate Up-Conversion for High Efficiency Video Coding

Machine Learning-Based H.264/AVC to HEVC Transcoding Via Motion Information Reuse and Coding Mode Similarity Analysis.

An Extended Skip Strategy for Inter Prediction

Enhanced Motion-Compensated Video Coding with Deep Virtual Reference Frame Generation

Learned Bi-Directional Motion Prediction for Video Compression.

Locally Refined Motion Compensation for Future Video Coding

Deep Affine Motion Compensation Network for Inter Prediction in VVC

Motion Blur Compensation In Scalable Hevc Hybrid Video Coding

An Adaptive Linear Estimator Based Approach to Bi-Directional Motion Compensated Prediction

Deep Frame Prediction for Video Coding

Multi-Scale Convolutional Neural Network-Based Intra Prediction for Video Coding.

Convolutional Neural Network-Based Coefficients Prediction for HEVC Intra-Predicted Residues

Bi-prediction Enhancement with Deep Frame Prediction Network for Versatile Video Coding

Optimized Spatial Recurrent Network for Intra Prediction in Video Coding

Multi-Hypothesis Prediction Based on Implicit Motion Vector Derivation for Video Coding.

Deep Convolutional Neural Network For Decompressed Video Enhancement

Advanced CNN Based Motion Compensation Fractional Interpolation

Neural Network Based Inter Prediction for HEVC