Abstract:As the upcoming video coding standard, Versatile Video Coding (i.e., VVC) achieves up to 30% Bjøntegaard delta bit-rate (BD-rate) reduction compared with High Efficiency Video Coding (H.265/HEVC). To eliminate or alleviate different kinds of compression artifacts like blocking, ringing, blurring and contouring effects, three in-loop filters, i.e. de-blocking filter (DBF), sample adaptive offset (SAO) and adaptive loop filter (ALF), have been involved in VVC. Recently, Convolutional Neural Network (CNN) has attracted tremendous attention and shows great potential in many tasks in image processing. In this work, we design a CNN-based in-loop filter as an integrated single-model solution which is adaptive to almost any scenarios in video coding. An architecture named as ADCNN (i.e., Attention based Dual-scale CNN) with an attention based processing block is proposed to reduce artifacts of I frames and B frames, which take advantage of informative priors such as the quantization parameter (QP) and partitioning information. Different from existing CNN-based filtering methods, which are mainly designed for the luma component and may need to train different models for different QPs, the proposed filter is adapted to different QPs and different frame types, and all the components (i.e., both luma and chroma) are processed simultaneously with feature exchange and fusion between components for information supplementary. Experimental results show that the proposed ADCNN filter can achieve 6.54%, 13.27%, 15.72% BD-rate savings for Y, U, V respectively under the all intra configuration and 2.81%, 7.86%, 8.60% BD-rate savings under the random access configuration. It can be used to replace all the conventional in-loop filters and also outperforms them without increase in encoding time.

Adaptive CNN-Enhanced In-Loop Filter for Lossy Video Coding.

Attention-Based Dual-Scale CNN In-Loop Filter for Versatile Video Coding.

Content-Aware Convolutional Neural Network for In-Loop Filtering in High Efficiency Video Coding

A CNN-Based In-Loop Filter with CU Classification for HEVC

An Integrated CNN-based Post Processing Filter For Intra Frame in Versatile Video Coding

Lightweight CNN-Based In-Loop Filter for VVC Intra Coding

NR-CNN: Nested-Residual Guided CNN In-loop Filtering for Video Coding

A Learning-Based Low Complexity In-Loop Filter for Video Coding

Low-Rank-Based Nonlocal Adaptive Loop Filter for High-Efficiency Video Compression

A CU-level Adaptive Decision Method for CNN-based In-Loop Filtering

Multi-Density Convolutional Neural Network for In-Loop Filter in Video Coding.

Deep Convolutional Neural Network For Decompressed Video Enhancement

Convolutional Neural Network Based In-Loop Filter for VVC Intra Coding

CNN-Based Sample Adaptive Offset Optimization in HEVC for Streaming Video

Dense Residual Convolutional Neural Network Based In-Loop Filter for HEVC

A Nonlocal HEVC In-Loop Filter Using CNN-based Compression Noise Estimation

Multi-Density Attention Network for Loop Filtering in Video Compression

Enhanced Bi-Prediction with Convolutional Neural Network for High-Efficiency Video Coding

Recursive Residual Convolutional Neural Network- Based In-Loop Filtering for Intra Frames

Multi-Gradient Convolutional Neural Network Based In-Loop Filter For Vvc

One-for-All: an Efficient Variable Convolution Neural Network for In-Loop Filter of VVC