Abstract:High-Efficiency Video Coding (HEVC) is the latest video coding standard developed by the Joint Collaborative Team on Video Coding (JCT-VC). An effective rate control mechanism plays a critical role in making the best possible use of the network resources. The coding performance can be maximised through the appropriate allocation of bits under the constraints of a total bit rate budget and the buffer size. Existing rate control strategies generally determine the rate control parameters according to previous encoding results. However, there is no priori-knowledge available when the first frame of a video sequence is encoded or scene changes occur. Consequently, it is challenging to choose appropriate rate control parameters in such cases. In order to address this issue, we investigated deep learning oriented methods and proposed a Convolutional Neural Network (CNN)-based rate control scheme for HEVC intra-coding. As the CNN has the advantage of local perception and the Coding Tree Unit's (CTU's) Quantisation Parameter (QP) value is determined during the CTU level rate control process, we applied the CNN at the CTU level as well to achieve an accurate QP prediction. An improved QP determination model (R-QP model) involving only two model coefficients (µ and v) was suggested to simplify the QP calculation process. A CNN was used to predict the model coefficients, namely µ and v. Our CNN has four convolution layers, two pooling layers and three fully connected layers. Each of the convolution layers contains a Rectified Linear Unit (ReLU). The last fully connected layer outputs the predicted values of µ and v. The images from the UCID dataset and the RAISE dataset were used to train the CNN. In order to obtain a universal model, none of the frames in the HEVC common test sequences were employed as training images. In order to achieve a reasonable bit budget allocation, a CNN-based framework was used to predict the bit consumption for each CTU according to the entire picture content that it contains. At the training stage, the network took the original images and the QP values as inputs, and the corresponding actual bit consumptions were used as the labels. Once the CNN coefficients have been obtained through training, the bit consumption for each CTU under different QP values can be directly predicted from the original images. The predicted bit consumption was subsequently used to readjust the bit budget for each CTU to achieve a more reasonable bit allocation. We implemented the proposed CNN-based rate control scheme in the HEVC reference software HM 16.9. As the priori-knowledge of the previous encoding results had not been considered in the proposed algorithm, only the first frame of each video sequence was encoded. The experimental results showed that our proposed scheme performs better than the default rate control implementation in the HEVC reference software, namely the BDBR is reduced by 1.33% on average while maintaining the same reconstructed picture quality. Compared with the state-of-the-art CNN-based rate control algorithm proposed by Li, our schemes achieved better coding efficiency. It can be concluded that the proposed R-QP model is effective in describing the rate-distortion relationship in HEVC, and the CNN-based bit allocation scheme is appropriate for ensuring that the best use of each bit is made.

Learning-Based Rate Control for Video-Based Point Cloud Compression

Occupancy-Map-Based Rate Distortion Optimization and Partition for Video-Based Point Cloud Compression

Rate Control for Geometry-Based LiDAR Point Cloud Compression Via Multi-Factor Modeling

Rate-Distortion-Guided Learning Approach with Cross-Projection Information for V-PCC Fast CU Decision

Content-aware Rate Control for Geometry-based Point Cloud Compression

Neural Rate Control for Learned Video Compression

Perceptually Weighted Rate Distortion Optimization for Video-Based Point Cloud Compression

Low Complexity Coding Unit Decision for Video-Based Point Cloud Compression.

Occupancy Map Guided Fast Video-Based Dynamic Point Cloud Coding

Frame-level Rate Control for Geometry-based LiDAR Point Cloud Compression

Λ-Domain VVC Rate Control Based on Nash Equilibrium

Spatially Scalable Video-Based Point Cloud Compression.

Rate-Distortion Optimized Skip Coding of Region Adaptive Hierarchical Transform Coefficients for MPEG G-PCC

Real-time Scene-aware LiDAR Point Cloud Compression Using Semantic Prior Representation

A Convolutional Neural Network-Based Approach to Rate Control in Hevc Intra Coding

Occupancy map-based low complexity motion prediction for video-based point cloud compression

Rate-Quality Based Rate Control Model for Neural Video Compression.

Convolutional Neural Network-based Occupancy Map Accuracy Improvement for Video-based Point Cloud Compression

Rate-Distortion Optimal Transform Coefficient Selection for Unoccupied Regions in Video-Based Point Cloud Compression

A Rate Control Scheme for HEVC Intra Coding Using Convolution Neural Network (CNN)

Cross-Modal Distortion Approximation for Fast Bit Allocation of Video-Based Point Cloud Compression