Abstract:Motion model based video coding approach, which employs sparse sets of keypoints instead of dense optical flows, can efficiently compress videos at ultra-low bitrates. Such schemes obtain notable performance gains over traditional video codecs in face-centric scenarios, such as video conferencing. However, due to the high complexity of human poses, there is still a lack of research on motion model based human body video coding, especially in the case of large pose variations. In order to overcome this limitation, we present a thin-plate spline motion model based portrait video compression framework oriented to adaptive pose processing. Firstly a more flexible thin-plate spline transformation rather than simple affine transformation is adopted for motion estimation, since the nonlinear property allows representing more complex motions. Meanwhile, spatial constraints are incorporated into the keypoint detector to generate keypoints that are more consistent with the human poses, thus obtaining more accurate optical flow. In addition, a motion intensity evaluation module is designed at the encoder side to dynamically evaluate the inter-frame motion intensity. Adaptive Reference Frame Selection algorithm is then further devised at the decoder side to adaptively select the reconstruction scheme for different intensities of portrait motion. Finally, a multi-frame reconstruction module is introduced for large pose variations to improve the consistency of human pose and subjective quality. The experimental results demonstrate that compared to the state-of-the-art video coding standard Versatile Video Coding and existing motion model based compression techniques, our proposed scheme can better cope with large pose variation scenarios and outperforms in both objective and subjective quality at the similar bitrate with higher temporal consistency.

Optimization-based key frame extraction for motion capture animation

New Fusional Framework Combining Sparse Selection and Clustering for Key Frame Extraction.

A Novel Compact Yet Rich Key Frame Creation Method for Compressed Video Summarization

An Efficient Keyframe Extraction from Motion Capture Data

Key frame extraction from motion capture data based on gpu

Learning a Deep Motion Interpolation Network for Human Skeleton Animations

Extraction of Keyframe of Motion Capture Data Based on Layered Curve Simplification

Key-Frame Extraction from Motion Capture Data

An optimization method of human skeleton keyframes selection for action recognition

A 3D Human Motion Refinement Method Based on Sparse Motion Bases Selection.

Adaptive Skeleton-Driven Cages for Mesh Sequences

Long‐term Motion In‐betweening via Keyframe Prediction

MoCap-Solver: A Neural Solver for Optical Motion Capture Data

Motion Keyframe Interpolation for Any Human Skeleton via Temporally Consistent Point Cloud Sampling and Reconstruction

Predicting Missing Markers In Human Motion Capture Using L1-Sparse Representation

A novel video abstraction method based on fast clustering of the regions of interest in key frames

Model-based portrait video compression with spatial constraint and adaptive pose processing

Inverse kinematics using dynamic joint parameters: inverse kinematics animation synthesis learnt from sub-divided motion micro-segments

MoCap-solver

Key frame extraction based on global motion statistics for team-sport videos

Facial animation by optimized blendshapes from motion capture data