Abstract:The Versatile Video Coding (VVC) standard introduces a block partitioning structure known as quadtree plus nested multi-type tree (QTMTT), which allows more flexible block partitioning compared to its predecessors, like High Efficiency Video Coding (HEVC). Meanwhile, the partition search (PS) process, which is to find out the best partitioning structure for optimizing the rate-distortion cost, becomes far more complicated for VVC than for HEVC. Also, the PS process in VVC reference software (VTM) is not friendly to hardware implementation. We propose a partition map prediction method for fast block partitioning in VVC intra-frame encoding. The proposed method may replace PS totally or be combined with PS partially, thereby achieving adjustable acceleration of the VTM intra-frame encoding. Different from the previous methods for fast block partitioning, we propose to represent a QTMTT-based block partitioning structure by a partition map, which consists of a quadtree (QT) depth map, several multi-type tree (MTT) depth maps, and several MTT direction maps. We then propose to predict the optimal partition map from the pixels through a convolutional neural network (CNN). We propose a CNN structure, known as Down-Up-CNN, for the partition map prediction, where the CNN structure emulates the recursive nature of the PS process. Moreover, we design a post-processing algorithm to adjust the network output partition map, so as to obtain a standard-compliant block partitioning structure. The post-processing algorithm may produce a partial partition tree as well; then based on the partial partition tree, the PS process is performed to obtain the full tree. Experimental results show that the proposed method achieves to encoding acceleration for the VTM-10.0 intra-frame encoder, with the ratio de- ending on how much PS is performed. Especially, when achieving encoding acceleration, the compression efficiency loss is 2.77% in BD-rate, which is a better tradeoff than the previous methods.

Effective VVC Intra Prediction Based on Ensemble Learning

Multi-scale and Bi-path Method Based on Image Entropy and CNN for Fast CU Partition in VVC

Faster Intra-Prediction of Versatile Video Coding Using a Concatenate-Designed CNN via DCT Coefficients

Adaptive CU Split Decision with Pooling-variable CNN for VVC Intra Encoding.

Spatio-Temporal Convolutional Neural Network for Enhanced Inter Prediction in Video Coding

CNN-based Partitioning Structure Prediction for VVC Intra Speedup: Bottom-Up-based and Top-Down-based.

Convolutional Neural Network-based Split Prediction for VVC Intra Speedup

Width-Adaptive CNN: Fast CU Partition Prediction for VVC Screen Content Coding

Efficient Intra Coding Through Hierarchical CU Partition Prediction for VVC.

ResNet-Based Fast CU Partition Decision Algorithm for VVC

A Fast Decision Algorithm for VVC Intra-Coding Based on Texture Feature and Machine Learning

Light-weight CNN-based VVC Inter Partitioning Acceleration

Texture and Correlation Based Fast Intra Prediction Algorithm for HEVC

Cnn-Based Depth Map Prediction for Fast Block Partitioning in HEVC Intra Coding.

Optimized Spatial Recurrent Network for Intra Prediction in Video Coding

Effective CU size decision for HEVC intracoding.

Partition Map Prediction for Fast Block Partitioning in VVC Intra-Frame Coding

Intra Block Partition Structure Prediction Via Convolutional Neural Network.

A Method to Reduce the Intra-Frame Prediction Complexity of HEVC Based on D-CNN

Enhanced Intra Prediction with Recurrent Neural Network in Video Coding

HG-FCN: Hierarchical Grid Fully Convolutional Network for Fast VVC Intra Coding