Abstract:High-Efficiency Video Coding (HEVC) is the latest video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). An effective rate control mechanism plays a critical role in making the best possible use of the network resources. The coding performance can be maximised through the appropriate allocation of bits under the constraints of a total bit rate budget and the buffer size. Existing rate control strategies generally determine the rate control parameters according to previous encoding results. However, there is no priori-knowledge available when the first frame of a video sequence is encoded or scene changes occur. Consequently, it is challenging to choose appropriate rate control parameters in such cases. In order to address this issue, we investigated deep learning oriented methods and proposed a Convolutional Neural Network (CNN)-based rate control scheme for HEVC intra-coding. As the CNN has the advantage of local perception and the Coding Tree Unit's (CTU's) Quantisation Parameter (QP) value is determined during the CTU level rate control process, we applied the CNN at the CTU level as well to achieve an accurate QP prediction. An improved QP determination model (R-QP model) involving only two model coefficients (µ and v) was suggested to simplify the QP calculation process. A CNN was used to predict the model coefficients, namely µ and v. Our CNN has four convolution layers, two pooling layers and three fully connected layers. Each of the convolution layers contains a Rectified Linear Unit (ReLU). The last fully connected layer outputs the predicted values of µ and v. The images from the UCID dataset and the RAISE dataset were used to train the CNN. In order to obtain a universal model, none of the frames in the HEVC common test sequences were employed as training images. In order to achieve a reasonable bit budget allocation, a CNN-based framework was used to predict the bit consumption for each CTU according to the entire picture content that it contains. At the training stage, the network took the original images and the QP values as inputs, and the corresponding actual bit consumptions were used as the labels. Once the CNN coefficients have been obtained through training, the bit consumption for each CTU under different QP values can be directly predicted from the original images. The predicted bit consumption was subsequently used to readjust the bit budget for each CTU to achieve a more reasonable bit allocation. We implemented the proposed CNN-based rate control scheme in the HEVC reference software HM 16.9. As the priori-knowledge of the previous encoding results had not been considered in the proposed algorithm, only the first frame of each video sequence was encoded. The experimental results showed that our proposed scheme performs better than the default rate control implementation in the HEVC reference software, namely the BDBR is reduced by 1.33% on average while maintaining the same reconstructed picture quality. Compared with the state-of-the-art CNN-based rate control algorithm proposed by Li, our schemes achieved better coding efficiency. It can be concluded that the proposed R-QP model is effective in describing the rate-distortion relationship in HEVC, and the CNN-based bit allocation scheme is appropriate for ensuring that the best use of each bit is made.

A QP-adaptive Mechanism for CNN-based Filter in Video Coding

QA-Filter: A QP-Adaptive Convolutional Neural Network Filter for Video Coding

An Integrated CNN-based Post Processing Filter For Intra Frame in Versatile Video Coding

Content-Aware Convolutional Neural Network for In-Loop Filtering in High Efficiency Video Coding

An Efficient QP Variable Convolutional Neural Network Based In-loop Filter for Intra Coding

One-for-All: an Efficient Variable Convolution Neural Network for In-Loop Filter of VVC

A CU-level Adaptive Decision Method for CNN-based In-Loop Filtering

A CNN-based Prediction-Aware Quality Enhancement Framework for VVC

NR-CNN: Nested-Residual Guided CNN In-loop Filtering for Video Coding

A Rate Control Scheme for HEVC Intra Coding Using Convolution Neural Network (CNN)

Adaptive CU Split Decision with Pooling-variable CNN for VVC Intra Encoding.

QCGFNet: A Lightweight Color Image Denoising Network

A CNN-Based In-Loop Filter with CU Classification for HEVC

An explicit self-attention-based multimodality CNN in-loop filter for versatile video coding

Adaptive QP Cascading Algorithm for Scalable Video Coding

An Efficient Low-Complexity Convolutional Neural Network Filter

Adaptive Global Power-of-Two Ternary Quantization Algorithm Based on Unfixed Boundary Thresholds

Adaptive Initial Quantization Parameter Determination for H.264/AVC Video Transcoding

An Adaptive Perceptual Quantization Algorithm Based on Block-Level JND for Video Coding

A novel adaptive quantization method for video coding

A Learning-Based Low Complexity In-Loop Filter for Video Coding