DMVC: Decomposed Motion Modeling for Learned Video Compression
Kai Lin,Chuanmin Jia,Xinfeng Zhang,Shanshe Wang,Siwei Ma,Wen Gao
DOI: https://doi.org/10.1109/tcsvt.2022.3233221
IF: 5.859
2023-01-01
IEEE Transactions on Circuits and Systems for Video Technology
Abstract:Inter prediction is the critical component in hybrid coding framework to deal with the temporal redundancy. Most of the neural video coding methods typically follow the motion compensation based inter coding scheme, establishing motion vector (MV) as the central role. In this paper, we innovatively propose an efficient motion modeling approach by inherently decomposing it into two components, the intrinsic motion and the compensatory motion. The intrinsic motion originates from the implicit spatiotemporal context hidden in the historical sequence, which can be intuitively captured free of bits. On the top of it, the compensatory motion acts a role of structural refinement and texture enhancement as a form of side information. In particular, the inter prediction is performed in the feature space as a manner of progressive temporal transition, conditioned on the decomposed motion. By the motion decomposition paradigm, we innovatively answer the question of motion representation, compensation and coding in the learned video compression framework. With the temporal prediction, the remaining pixel residue is signaled to obtain the reconstruction. Extensive experimental results demonstrate that the proposed method achieves state-of-the-art coding performance on par with other end-to-end coding methods, and outperforms versatile video coding (VVC) under low-delay P (LDP) configuration in terms of MS-SSIM metric.