Abstract:In-loop filtering (ILF) is a key technology for removing the artifacts in image/video coding standards. Recently, neural network-based in-loop filtering methods achieve remarkable coding gains beyond the capability of advanced video coding standards, which becomes a powerful coding tool candidate for future video coding standards. However, the utilization of deep neural networks brings heavy time and computational complexity, and high demands of high-performance hardware, which is challenging to apply to the general uses of coding scene. To address this limitation, inspired by explorations in image restoration, we propose an efficient and practical in-loop filtering scheme by adopting the Look-up Table (LUT). We train the DNN of in-loop filtering within a fixed filtering reference range, and cache the output values of the DNN into a LUT via traversing all possible inputs. At testing time in the coding process, the filtered pixel is generated by locating input pixels (to-be-filtered pixel with reference pixels) and interpolating cached filtered pixel values. To further enable the large filtering reference range with the limited storage cost of LUT, we introduce the enhanced indexing mechanism in the filtering process, and clipping/finetuning mechanism in the training. The proposed method is implemented into the Versatile Video Coding (VVC) reference software, VTM-11.0. Experimental results show that the ultrafast, very fast, and fast mode of the proposed method achieves on average 0.13%/0.34%/0.51%, and 0.10%/0.27%/0.39% BD-rate reduction, under the all intra (AI) and random access (RA) configurations. Especially, our method has friendly time and computational complexity, only 101%/102%-104%/108% time increase with 0.13-0.93 kMACs/pixel, and only 164-1148 KB storage cost for a single model. Our solution may shed light on the journey of practical neural network-based coding tool evolution.

A New HEVC In-Loop Filter Based on Multi-channel Long-Short-Term Dependency Residual Networks

Nonlocal Adaptive In-Loop Filter Via Content-Dependent Soft-Thresholding for HEVC.

A Learning-Based Low Complexity In-Loop Filter for Video Coding

Multi-Gradient Convolutional Neural Network Based In-Loop Filter For Vvc

Improved Intra Boundary Filters for HEVC

Transform-Domain In-Loop Filter With Block Similarity For Hevc

A DenseNet Based Approach for Multi-Frame In-Loop Filter in HEVC

Structure-driven Adaptive Non-local Filter for High Efficiency Video Coding (HEVC).

Residual in Residual Based Convolutional Neural Network In-loop Filter for AVS3

An Integrated CNN-based Post Processing Filter For Intra Frame in Versatile Video Coding

Content-Aware Convolutional Neural Network for In-Loop Filtering in High Efficiency Video Coding

In-Loop Filtering via Trained Look-Up Tables

A CNN-Based In-Loop Filter with CU Classification for HEVC

Convolutional Neural Network Based In-Loop Filter for VVC Intra Coding

Deep In-Loop Filtering via Multi-Domain Correlation Learning and Partition Constraint for Multiview Video Coding

Implementation of In-Loop Filter for HEVC Decoder on Reconfigurable Processor

Neural Network Based In-Loop Filter with Constrained Memory

Residual Highway Convolutional Neural Networks for In-Loop Filtering in HEVC.

Combining Progressive Rethinking and Collaborative Learning: A Deep Framework for In-Loop Filtering

NR-CNN: Nested-Residual Guided CNN In-loop Filtering for Video Coding

Non-Local Structure-Based Filter for Video Coding