Abstract:Video compression is indispensable to most video analysis systems. Despite saving the transportation bandwidth, it also deteriorates downstream video understanding tasks, especially at low-bitrate settings. To systematically investigate this problem, we first thoroughly review the previous methods, revealing that three principles, i.e., task-decoupled, label-free, and data-emerged semantic prior, are critical to a machine-friendly coding framework but are not fully satisfied so far. In this paper, we propose a traditional-neural mixed coding framework that simultaneously fulfills all these principles, by taking advantage of both traditional codecs and neural networks (NNs). On one hand, the traditional codecs can efficiently encode the pixel signal of videos but may distort the semantic information. On the other hand, highly non-linear NNs are proficient in condensing video semantics into a compact representation. The framework is optimized by ensuring that a transportation-efficient semantic representation of the video is preserved w.r.t. the coding procedure, which is spontaneously learned from unlabeled data in a self-supervised manner. The videos collaboratively decoded from two streams (codec and NN) are of rich semantics, as well as visually photo-realistic, empirically boosting several mainstream downstream video analysis task performances without any post-adaptation procedure. Furthermore, by introducing the attention mechanism and adaptive modeling scheme, the video semantic modeling ability of our approach is further enhanced. Fianlly, we build a low-bitrate video understanding benchmark with three downstream tasks on eight datasets, demonstrating the notable superiority of our approach. All codes, data, and models will be open-sourced for facilitating future research.

A New Compressive Sensing Video Coding Framework Based on Gaussian Mixture Model

Efficient Compressive Sensing Video Compression Method Based On Gaussian Mixture Models

Compressive Sensing Depth Video Coding via Gaussian Mixture Models and Object Edges.

Distributed Depth Video Coding Based on Compressive Sensing and Gaussian Mixture Models

Coding Scheme for Compressive Sensing Depth Video Based on Adaptive Bits Allocation

Extended application of scalable video coding methods

Multi-View Video Coding Based on Vector Estimation and Weighted Disparity Interpolation

Multi-view Video Coding Based on Resultant Vector Estimation

A Predictive VQ Based Video Compression Scheme

Compressive Sensing Image Coding with Perceptual Weighting Measuring Matrix

A Compressive Prior Guided Mask Predictive Coding Approach for Video Analysis.

Video compression based on distributed source coding principles

An Emerging Coding Paradigm VCM: A Scalable Coding Approach Beyond Feature and Signal

A Coding Framework and Benchmark towards Compressed Video Understanding

A Novel Image/video Coding Method Based on Compressed Sensing Theory

Fast Video Coding Based on Gaussian Model of Dct Coefficients

Temporal Compressive Video Reconstruction Using Gaussian Scale Mixture Model

Video Codec Based on Compressed Sensing

An approach to the compression of residual data with GPCA in video coding

Increasing Compression Ratio of Low Complexity Compressive Sensing Video Encoder with Application-Aware Configurable Mechanism

A Coding Framework and Benchmark towards Low-Bitrate Video Understanding