Abstract:As the upcoming video coding standard, Versatile Video Coding (i.e., VVC) achieves up to 30% Bjøntegaard delta bit-rate (BD-rate) reduction compared with High Efficiency Video Coding (H.265/HEVC). To eliminate or alleviate different kinds of compression artifacts like blocking, ringing, blurring and contouring effects, three in-loop filters, i.e. de-blocking filter (DBF), sample adaptive offset (SAO) and adaptive loop filter (ALF), have been involved in VVC. Recently, Convolutional Neural Network (CNN) has attracted tremendous attention and shows great potential in many tasks in image processing. In this work, we design a CNN-based in-loop filter as an integrated single-model solution which is adaptive to almost any scenarios in video coding. An architecture named as ADCNN (i.e., Attention based Dual-scale CNN) with an attention based processing block is proposed to reduce artifacts of I frames and B frames, which take advantage of informative priors such as the quantization parameter (QP) and partitioning information. Different from existing CNN-based filtering methods, which are mainly designed for the luma component and may need to train different models for different QPs, the proposed filter is adapted to different QPs and different frame types, and all the components (i.e., both luma and chroma) are processed simultaneously with feature exchange and fusion between components for information supplementary. Experimental results show that the proposed ADCNN filter can achieve 6.54%, 13.27%, 15.72% BD-rate savings for Y, U, V respectively under the all intra configuration and 2.81%, 7.86%, 8.60% BD-rate savings under the random access configuration. It can be used to replace all the conventional in-loop filters and also outperforms them without increase in encoding time.

Non-Local Structure-Based Filter for Video Coding

Structure-driven Adaptive Non-local Filter for High Efficiency Video Coding (HEVC).

A Low Complexity Deblocking Filtering for Multiview Video Coding

Perceptually adaptive joint deringing - deblocking filtering for scalable video coding

Attention-Based Dual-Scale CNN In-Loop Filter for Versatile Video Coding.

An efficient deblocking filter algorithm in multi-view video coding

Nonlocal Adaptive In-Loop Filter Via Content-Dependent Soft-Thresholding for HEVC.

Nonlocal In Loop Filter: The Future Way Towards Next Generation Video Coding?

A Learning-Based Low Complexity In-Loop Filter for Video Coding

An Integrated CNN-based Post Processing Filter For Intra Frame in Versatile Video Coding

Parametric Non-local In-loop Filter for Future Video Coding

Deblocking Filtering Based on View Correlation for Multiview Video Decoding

Optimized Non-local In-Loop Filter for Video Coding

Nonlocal In-Loop Filter: The Way Toward Next-Generation Video Coding?

A New HEVC In-Loop Filter Based on Multi-channel Long-Short-Term Dependency Residual Networks

Transform-Domain In-Loop Filter With Block Similarity For Hevc

Deformable Wiener Filter for Future Video Coding

A higher order transform domain filter exploiting non-local spatial correlation for video coding

In-loop Deblocking Filter for Block Based Video Coding

De-ringing Filter for Scalable Video Coding

Low-Rank-Based Nonlocal Adaptive Loop Filter for High-Efficiency Video Compression