Abstract:In-loop filtering (ILF) is a key technology for removing the artifacts in image/video coding standards. Recently, neural network-based in-loop filtering methods achieve remarkable coding gains beyond the capability of advanced video coding standards, which becomes a powerful coding tool candidate for future video coding standards. However, the utilization of deep neural networks brings heavy time and computational complexity, and high demands of high-performance hardware, which is challenging to apply to the general uses of coding scene. To address this limitation, inspired by explorations in image restoration, we propose an efficient and practical in-loop filtering scheme by adopting the Look-up Table (LUT). We train the DNN of in-loop filtering within a fixed filtering reference range, and cache the output values of the DNN into a LUT via traversing all possible inputs. At testing time in the coding process, the filtered pixel is generated by locating input pixels (to-be-filtered pixel with reference pixels) and interpolating cached filtered pixel values. To further enable the large filtering reference range with the limited storage cost of LUT, we introduce the enhanced indexing mechanism in the filtering process, and clipping/finetuning mechanism in the training. The proposed method is implemented into the Versatile Video Coding (VVC) reference software, VTM-11.0. Experimental results show that the ultrafast, very fast, and fast mode of the proposed method achieves on average 0.13%/0.34%/0.51%, and 0.10%/0.27%/0.39% BD-rate reduction, under the all intra (AI) and random access (RA) configurations. Especially, our method has friendly time and computational complexity, only 101%/102%-104%/108% time increase with 0.13-0.93 kMACs/pixel, and only 164-1148 KB storage cost for a single model. Our solution may shed light on the journey of practical neural network-based coding tool evolution.

Nonuniform Filter Banks for Audio Encoding

A New Time-Frequency Mapping Structure for High Quality Audio Coding.

Gain comparison of time-frequency mapping for high quality audio coding

Simple and Efficient Loop-Filtering for Low Bit-Rate Coding

An Audio Compression Algorithm Using a Wavelet Packet Transform

Learning Long-Term Filter Banks for Audio Source Separation and Audio Scene Classification

Parametric Non-local In-loop Filter for Future Video Coding

Linear Time-Frequency Music Algorithm

Subband Image Coding Using Filter Banks with Non-Uniform Passband Distribution

Filter bank property of direct fast iterative filtering and its applications

Optimized algorithm for synthesis filter banks in HDTV audio decoding and implementation on FPGA

Deformable Wiener Filter for Future Video Coding

SpecTNT: a Time-Frequency Transformer for Music Audio

Encoder-embedded Temporal-Spatial Wiener Filter for Video Encoding.

Enhanced Motion Compensated Temporal Filter for VVenC

In-Loop Filtering via Trained Look-Up Tables

Frequency-Aware Transformer for Learned Image Compression

Efficient Aperture Fill Time Correction for Wideband Sparse Array Using Improved Variable Fractional Delay Filters

Frequency domain variant of Velvet noise and its application to acoustic measurements

Optimized Adaptive Loop Filter in Versatile Video Coding.

Geometry Transformation-Based Adaptive In-Loop Filter