Abstract:As the upcoming video coding standard, Versatile Video Coding (i.e., VVC) achieves up to 30% Bjøntegaard delta bit-rate (BD-rate) reduction compared with High Efficiency Video Coding (H.265/HEVC). To eliminate or alleviate different kinds of compression artifacts like blocking, ringing, blurring and contouring effects, three in-loop filters, i.e. de-blocking filter (DBF), sample adaptive offset (SAO) and adaptive loop filter (ALF), have been involved in VVC. Recently, Convolutional Neural Network (CNN) has attracted tremendous attention and shows great potential in many tasks in image processing. In this work, we design a CNN-based in-loop filter as an integrated single-model solution which is adaptive to almost any scenarios in video coding. An architecture named as ADCNN (i.e., Attention based Dual-scale CNN) with an attention based processing block is proposed to reduce artifacts of I frames and B frames, which take advantage of informative priors such as the quantization parameter (QP) and partitioning information. Different from existing CNN-based filtering methods, which are mainly designed for the luma component and may need to train different models for different QPs, the proposed filter is adapted to different QPs and different frame types, and all the components (i.e., both luma and chroma) are processed simultaneously with feature exchange and fusion between components for information supplementary. Experimental results show that the proposed ADCNN filter can achieve 6.54%, 13.27%, 15.72% BD-rate savings for Y, U, V respectively under the all intra configuration and 2.81%, 7.86%, 8.60% BD-rate savings under the random access configuration. It can be used to replace all the conventional in-loop filters and also outperforms them without increase in encoding time.

Residual in Residual Based Convolutional Neural Network In-loop Filter for AVS3

An Attention Based CNN with Temporal Hierarchical Deployment for AVS3 Inter In-loop Filtering

Coordinate Attention-Based Convolution Neural Network for In-loop Filter of AVS3

Attention-Based Dual-Scale CNN In-Loop Filter for Versatile Video Coding.

NR-CNN: Nested-Residual Guided CNN In-loop Filtering for Video Coding

Convolutional Neural Network Based In-Loop Filter for VVC Intra Coding

Low-Complexity Multi-Model CNN In-Loop Filter for AVS3

CNN-Based Inter Prediction Refinement for AVS3

Towards Next Generation Video Coding: from Neural Network Based Predictive Coding to In-Loop Filtering

Content-Aware Convolutional Neural Network for In-Loop Filtering in High Efficiency Video Coding

Lightweight CNN-Based In-Loop Filter for VVC Intra Coding

Multi-Gradient Convolutional Neural Network Based In-Loop Filter For Vvc

An Integrated CNN-based Post Processing Filter For Intra Frame in Versatile Video Coding

Recursive Residual Convolutional Neural Network- Based In-Loop Filtering for Intra Frames

A Learning-Based Low Complexity In-Loop Filter for Video Coding

Multi-Density Convolutional Neural Network for In-Loop Filter in Video Coding.

Dense Residual Convolutional Neural Network Based In-Loop Filter for HEVC

Joint Luma and Chroma Multi-Scale CNN In-loop Filter for Versatile Video Coding

A New HEVC In-Loop Filter Based on Multi-channel Long-Short-Term Dependency Residual Networks

An Efficient Low-Complexity Convolutional Neural Network Filter

One-for-All: an Efficient Variable Convolution Neural Network for In-Loop Filter of VVC