Abstract:As the upcoming video coding standard, Versatile Video Coding (i.e., VVC) achieves up to 30% Bjøntegaard delta bit-rate (BD-rate) reduction compared with High Efficiency Video Coding (H.265/HEVC). To eliminate or alleviate different kinds of compression artifacts like blocking, ringing, blurring and contouring effects, three in-loop filters, i.e. de-blocking filter (DBF), sample adaptive offset (SAO) and adaptive loop filter (ALF), have been involved in VVC. Recently, Convolutional Neural Network (CNN) has attracted tremendous attention and shows great potential in many tasks in image processing. In this work, we design a CNN-based in-loop filter as an integrated single-model solution which is adaptive to almost any scenarios in video coding. An architecture named as ADCNN (i.e., Attention based Dual-scale CNN) with an attention based processing block is proposed to reduce artifacts of I frames and B frames, which take advantage of informative priors such as the quantization parameter (QP) and partitioning information. Different from existing CNN-based filtering methods, which are mainly designed for the luma component and may need to train different models for different QPs, the proposed filter is adapted to different QPs and different frame types, and all the components (i.e., both luma and chroma) are processed simultaneously with feature exchange and fusion between components for information supplementary. Experimental results show that the proposed ADCNN filter can achieve 6.54%, 13.27%, 15.72% BD-rate savings for Y, U, V respectively under the all intra configuration and 2.81%, 7.86%, 8.60% BD-rate savings under the random access configuration. It can be used to replace all the conventional in-loop filters and also outperforms them without increase in encoding time.

An Efficient Low-Complexity Convolutional Neural Network Filter

A Convolutional Neural Network-Based Low Complexity Filter

A Learning-Based Low Complexity In-Loop Filter for Video Coding

Attention-Based Dual-Scale CNN In-Loop Filter for Versatile Video Coding.

Low Complexity In-Loop Filter for VVC Based on Convolution and Transformer

Multi-Gradient Convolutional Neural Network Based In-Loop Filter For Vvc

Low-Complexity Multi-Model CNN In-Loop Filter for AVS3

Content-Aware Convolutional Neural Network for In-Loop Filtering in High Efficiency Video Coding

A CNN-Based In-Loop Filter with CU Classification for HEVC

Recursive Residual Convolutional Neural Network- Based In-Loop Filtering for Intra Frames

Lightweight CNN-Based In-Loop Filter for VVC Intra Coding

An Integrated CNN-based Post Processing Filter For Intra Frame in Versatile Video Coding

Dense Residual Convolutional Neural Network Based In-Loop Filter for HEVC

One-for-All: an Efficient Variable Convolution Neural Network for In-Loop Filter of VVC

Residual in Residual Based Convolutional Neural Network In-loop Filter for AVS3

Adaptive CNN-Enhanced In-Loop Filter for Lossy Video Coding.

Convolutional Neural Network Based In-Loop Filter for VVC Intra Coding

Multi-Density Convolutional Neural Network for In-Loop Filter in Video Coding.

NR-CNN: Nested-Residual Guided CNN In-loop Filtering for Video Coding

Dynamic Convolutional Capsule Network for In-loop Filtering in HEVC Video Codec

Improving Efficiency in Convolutional Neural Network with Multilinear Filters