Temporal context video compression with flow-guided feature prediction
Yiming Wang,Qian Huang,Bin Tang,Huashan Sun,Xiaotong Guo,Zhuang Miao
DOI: https://doi.org/10.1016/j.eswa.2024.123322
IF: 8.5
2024-02-02
Expert Systems with Applications
Abstract:Over the past few years, data-driven video compression has achieved encouraging results and attracted increasing attention. However, previous works rely on feature space operations and residual coding, where feature space operations may degrade reconstructed frame quality due to estimated offset maps overflow, and residual coding uses a simple subtraction operation to remove redundancy that cannot explore the temporal correlation effectively. Also, error propagation is a common problem owing to the accumulation of reconstruction errors in inter-frame coding. To solve these issues, we propose an efficient video compression scheme in this paper. First, we propose a flow-guided feature prediction module to directly use optical flow for explicit guidance in the feature space. Second, we propose a temporal context compression module to replace the residual coding, which explores the hierarchical prior and temporal prior information and fuses them to boost the coding performance. Third, we further propose a three-stage training strategy to take advantage of single-frame and multi-frame information to avoid error propagation. Comprehensive experiments demonstrate that our proposed method achieves higher coding performance than the existing learning-based schemes and surpasses the current advanced coding standard (H.266/VVC) with low latency P frame configuration in MS-SSIM metric.
computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science