Abstract:As the upcoming video coding standard, Versatile Video Coding (i.e., VVC) achieves up to 30% Bjøntegaard delta bit-rate (BD-rate) reduction compared with High Efficiency Video Coding (H.265/HEVC). To eliminate or alleviate different kinds of compression artifacts like blocking, ringing, blurring and contouring effects, three in-loop filters, i.e. de-blocking filter (DBF), sample adaptive offset (SAO) and adaptive loop filter (ALF), have been involved in VVC. Recently, Convolutional Neural Network (CNN) has attracted tremendous attention and shows great potential in many tasks in image processing. In this work, we design a CNN-based in-loop filter as an integrated single-model solution which is adaptive to almost any scenarios in video coding. An architecture named as ADCNN (i.e., Attention based Dual-scale CNN) with an attention based processing block is proposed to reduce artifacts of I frames and B frames, which take advantage of informative priors such as the quantization parameter (QP) and partitioning information. Different from existing CNN-based filtering methods, which are mainly designed for the luma component and may need to train different models for different QPs, the proposed filter is adapted to different QPs and different frame types, and all the components (i.e., both luma and chroma) are processed simultaneously with feature exchange and fusion between components for information supplementary. Experimental results show that the proposed ADCNN filter can achieve 6.54%, 13.27%, 15.72% BD-rate savings for Y, U, V respectively under the all intra configuration and 2.81%, 7.86%, 8.60% BD-rate savings under the random access configuration. It can be used to replace all the conventional in-loop filters and also outperforms them without increase in encoding time.

A Nonlocal HEVC In-Loop Filter Using CNN-based Compression Noise Estimation

A CNN-Based In-Loop Filter with CU Classification for HEVC

Content-Aware Convolutional Neural Network for In-Loop Filtering in High Efficiency Video Coding

Attention-Based Dual-Scale CNN In-Loop Filter for Versatile Video Coding.

A Quality Enhancement Framework with Noise Distribution Characteristics for High Efficiency Video Coding

Nonlocal Adaptive In-Loop Filter Via Content-Dependent Soft-Thresholding for HEVC.

CNN-Based Sample Adaptive Offset Optimization in HEVC for Streaming Video

Spatial-Temporal Fusion Convolutional Neural Network for Compressed Video Enhancement in HEVC

Residual Highway Convolutional Neural Networks for In-Loop Filtering in HEVC.

Multi-Gradient Convolutional Neural Network Based In-Loop Filter For Vvc

A CNN-based Coding Unit Partition in HEVC for Video Processing.

Low-Rank-Based Nonlocal Adaptive Loop Filter for High-Efficiency Video Compression

Transform-Domain In-Loop Filter With Block Similarity For Hevc

One-for-All: an Efficient Variable Convolution Neural Network for In-Loop Filter of VVC

An Integrated CNN-based Post Processing Filter For Intra Frame in Versatile Video Coding

A CU-level Adaptive Decision Method for CNN-based In-Loop Filtering

A Learning-Based Low Complexity In-Loop Filter for Video Coding

A Lightweight CNN for Low-Complexity HEVC Intra Encoder

Enhancing Quality for HEVC Compressed Videos

A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding

Enhancing HEVC Compressed Videos with a Partition-Masked Convolutional Neural Network