Abstract:Animation production workflows centred around motion capture techniques often require animators to edit the motion for various artistic and technical reasons. This process generally uses a set of keyframes. Unsupervised keyframe selection methods for motion capture sequences are highly demanded to reduce the laborious annotations. However, most existing methods are optimization-based, which cause the issues of flexibility and efficiency and eventually constrains the interactions and controls with animators. To address these limitations, we propose a novel graph based deep reinforcement learning method for efficient unsupervised keyframe selection. First, a reward function is devised in terms of reconstruction difference by comparing the original sequence and the interpolated sequence produced by the keyframes. The reward complies with the requirements of the animation pipeline satisfying: 1) incremental reward to evaluate the interpolated keyframes immediately; 2) order insensitivity for consistent evaluation; and 3) non-diminishing return for comparable rewards between optimal and sub-optimal solutions. Then by representing each skeleton frame as a graph, a graph-based deep agent is guided to heuristically select keyframes to maximize the reward. During the inference it is no longer necessary to estimate the reconstruction difference, and the evaluation time can be reduced significantly. The experimental results on the CMU Mocap dataset demonstrate that our proposed method is able to select keyframes at a high efficiency without clearly compromising the quality in comparison with the state-of-the-art methods.

Key Frames Extraction from Human Motion Capture Data Based on Hybrid Particle Swarm Optimization Algorithm

Optimization-based key frame extraction for motion capture animation

Keyframe Extraction From Human Motion Capture Data Based On A Multiple Population Genetic Algorithm

New Fusional Framework Combining Sparse Selection and Clustering for Key Frame Extraction.

Extraction of Key-frame from Motion Capture Data Based on Pre-selection and Reconstruction Error Optimization

Key frame extraction from motion capture data based on gpu

Dynamic Programming Based Key-frame Extraction Algorithm

An Efficient Method of Key-Frame Extraction Based on a Cluster Algorithm

Motion Key-Frame Extraction By Using Optimized T-Stochastic Neighbor Embedding

An Efficient Keyframe Extraction from Motion Capture Data

Keyframe Extraction from Motion Capture Sequences with Graph based Deep Reinforcement Learning

Key Frames Hysteresis-Seeking Based on Motion Change Points for Rgb-D Video

Key-Frame Extraction from Motion Capture Data

An optimization method of human skeleton keyframes selection for action recognition

Motion Compensated Frame Interpolation Based on Mixed Particle Filter and Genetic Algorithm Approach

A novel video abstraction method based on fast clustering of the regions of interest in key frames

Technique of key-frame extraction and motion editing based on wavelet transform

A 3D Human Motion Refinement Method Based on Sparse Motion Bases Selection.

Accurate Key Frame Extraction Algorithm of Video Action for Aerobics Online Teaching

Motion Key-frames extraction based on amplitude of distance characteristic curve

Adaptive Key Frame Extraction Using Dynamic Programming