Abstract:Learning-based Neural Video Codecs (NVCs) have emerged as a compelling alternative to the standard video codecs, demonstrating promising performance, and simple and easily maintainable pipelines. However, NVCs often fall short of compression performance and occasionally exhibit poor generalization capability due to inference-only compression scheme and their dependence on training data. The instance-adaptive video compression techniques have recently been suggested as a viable solution, fine-tuning the encoder or decoder networks for a particular test instance video. However, fine-tuning all the model parameters incurs high computational costs, increases the bitrates, and often leads to unstable training. In this work, we propose a parameter-efficient instance-adaptive video compression framework. Inspired by the remarkable success of parameter-efficient fine-tuning on large-scale neural network models, we propose to use a lightweight adapter module that can be easily attached to the pretrained NVCs and fine-tuned for test video sequences. The resulting algorithm significantly improves compression performance and reduces the encoding time compared to the existing instant-adaptive video compression algorithms. Furthermore, the suggested fine-tuning method enhances the robustness of the training process, allowing for the proposed method to be widely used in many practical settings. We conducted extensive experiments on various standard benchmark datasets, including UVG, MCL-JVC, and HEVC sequences, and the experimental results have shown a significant improvement in rate-distortion (RD) curves (up to 5 dB PSNR improvements) and BD rates compared to the baselines NVC. Our code is available on <a class="link-external link-https" href="https://github.com/ohsngjun/PEVC" rel="external noopener nofollow">this https URL</a>}{<a class="link-external link-https" href="https://github.com/ohsngjun/PEVC" rel="external noopener nofollow">this https URL</a>.

Neural Video Compression with Re-Parametrisation Scene Content-Adaptive Network

Neural video compression using patio-temporal priors

Releasing the Parameter Latency of Neural Representation for High-Efficiency Video Compression

Neural Radiance Field-Assisted Static-Scene Video Coding

Neural Video Compression using Spatio-Temporal Priors

Group-aware Parameter-efficient Updating for Content-Adaptive Neural Video Compression

Implicit Neural Video Compression

Content Adaptive and Error Propagation Aware Deep Video Compression

Content Adaptive Latents and Decoder for Neural Image Compression

Overfitting the Data: Compact Neural Video Delivery Via Content-aware Feature Modulation

Instance-Adaptive Video Compression: Improving Neural Codecs by Training on the Test Set

Static-Scene Video Compression with Neural Radiance Fields

Image and Video Compression with Neural Networks: A Review

NeRFCodec: Neural Feature Compression Meets Neural Radiance Fields for Memory-Efficient Scene Representation

A Neural-network Enhanced Video Coding Framework beyond ECM

An End-to-End Learning Framework for Video Compression.

Deep Predictive Video Compression Using Mode-Selective Uni- and Bi-Directional Predictions Based on Multi-Frame Hypothesis

Scene Matters: Model-based Deep Video Compression

A Video Coding Method Based on Neural Network for CLIC2024

On Content-Aware Post-Processing: Adapting Statistically Learned Models to Dynamic Content

Parameter-Efficient Instance-Adaptive Neural Video Compression