Abstract:Recent years have witnessed rapid advances in learnt video coding. Most algorithms have solely relied on the vector-based motion representation and resampling (e.g., optical flow based bilinear sampling) for exploiting the inter frame redundancy. In spite of the great success of adaptive kernel-based resampling (e.g., adaptive convolutions and deformable convolutions) in video prediction for uncompressed videos, integrating such approaches with rate-distortion optimization for inter frame coding has been less successful. Recognizing that each resampling solution offers unique advantages in regions with different motion and texture characteristics, we propose a hybrid motion compensation (HMC) method that adaptively combines the predictions generated by these two approaches. Specifically, we generate a compound spatiotemporal representation (CSTR) through a recurrent information aggregation (RIA) module using information from the current and multiple past frames. We further design a one-to-many decoder pipeline to generate multiple predictions from the CSTR, including vector-based resampling, adaptive kernel-based resampling, compensation mode selection maps and texture enhancements, and combines them adaptively to achieve more accurate inter prediction. Experiments show that our proposed inter coding system can provide better motion-compensated prediction and is more robust to occlusions and complex motions. Together with jointly trained intra coder and residual coder, the overall learnt hybrid coder yields the state-of-the-art coding efficiency in low-delay scenario, compared to the traditional H.264/AVC and H.265/HEVC, as well as recently published learning-based methods, in terms of both PSNR and MS-SSIM metrics.

Content-Aware Layered Compound Video Compression

Compound Image Compression by Multi-Stage Prediction

Foreground-Background Parallel Compression with Residual Encoding for Surveillance Video

Content Adaptive and Error Propagation Aware Deep Video Compression

Lightweight content-adaptive coding in joint analyzing-encoding framework

A Joint Compression Scheme of Video Feature Descriptors and Visual Content.

A new scheme of layered coding of stereoscopic video

Temporal context video compression with flow-guided feature prediction

Deep Hierarchical Video Compression

Efficient Content Adaptive Plenoptic Video Coding

A Multi-View Video Coding Approach Using Layered Depth Image

Increasing Compression Ratio of Low Complexity Compressive Sensing Video Encoder with Application-Aware Configurable Mechanism

Flexible Video Coding Scheme for Content-Based Video Storage and Retrieval

A Compressive Prior Guided Mask Predictive Coding Approach for Video Analysis.

End-to-end Neural Video Coding Using a Compound Spatiotemporal Representation

Video data compaction coding method

Low-complexity Deep Video Compression with A Distributed Coding Architecture

Video Coding for Machines: A Paradigm of Collaborative Compression and Intelligent Analytics

Collaborative Scalable Visual Compression for Human-Centered Videos.

Extreme Generative Human-Oriented Video Coding Via Motion Representation Compression.

Video Object Analysis for Content Based Coding