Abstract:The task of reconstructing 3D human motion has wideranging applications. The gold standard Motion capture (MoCap) systems are accurate but inaccessible to the general public due to their cost, hardware and space constraints. In contrast, monocular human mesh recovery (HMR) methods are much more accessible than MoCap as they take single-view videos as inputs. Replacing the multi-view Mo- Cap systems with a monocular HMR method would break the current barriers to collecting accurate 3D motion thus making exciting applications like motion analysis and motiondriven animation accessible to the general public. However, performance of existing HMR methods degrade when the video contains challenging and dynamic motion that is not in existing MoCap datasets used for training. This reduces its appeal as dynamic motion is frequently the target in 3D motion recovery in the aforementioned applications. Our study aims to bridge the gap between monocular HMR and multi-view MoCap systems by leveraging information shared across multiple video instances of the same action. We introduce the Neural Motion (NeMo) field. It is optimized to represent the underlying 3D motions across a set of videos of the same action. Empirically, we show that NeMo can recover 3D motion in sports using videos from the Penn Action dataset, where NeMo outperforms existing HMR methods in terms of 2D keypoint detection. To further validate NeMo using 3D metrics, we collected a small MoCap dataset mimicking actions in Penn Action,and show that NeMo achieves better 3D reconstruction compared to various baselines.

Video-Interfaced Human Motion Capture Data Retrieval Based on the Normalized Motion Energy Image Representation.

Video-based human motion capture data retrieval via normalized motion energy image subspace projections

Video-Based Human Motion Capture Data Retrieval Via MotionSet Network

Generic Video-Based Motion Capture Data Retrieval

Performance-Driven Motion Retrieval and its Usability Evaluation

An efficient algorithm for content-based human motion retrieval

Motion Data Retrieval from Very Large Motion Databases

Vector Space Modeling and Retrieval of Human Motion Capture Data

Motion retrieval based on movement notation language: Motion Capture and Retrieval

A group of novel approaches and a toolkit for motion capture data reusing

Generic Content-Based Retrieval of Marker-Based Motion Capture Data.

An Efficient Sequence Retrieval Method Based on Motion Capture Data

Text-Like motion representation for human motion retrieval

Near-duplicate Keyframe Retrieval by Nonrigid Image Matching.

A Method Of Hyper-Sphere Cover In Multidimensional Space For Human Mocap Data Retrieval

Motion Retrieval Based On Movement Notation Language

A Retrieval Method for Human Mocap Data Based on Biomimetic Pattern Recognition

PESTA: an Elastic Motion Capture Data Retrieval Method

A Perceptual User Interface For Motion Retrieval

A Human Motion Retrieval Algorithm Based on Logical Similarity

NeMo: 3D Neural Motion Fields from Multiple Video Instances of the Same Action