Abstract:Accurate and reliable human motion reconstruction is crucial for creating natural interactions of full-body avatars in Virtual Reality (VR) and entertainment applications. As the Metaverse and social applications gain popularity, users are seeking cost-effective solutions to create full-body animations that are comparable in quality to those produced by commercial motion capture systems. In order to provide affordable solutions, though, it is important to minimize the number of sensors attached to the subject's body. Unfortunately, reconstructing the full-body pose from sparse data is a heavily under-determined problem. Some studies that use IMU sensors face challenges in reconstructing the pose due to positional drift and ambiguity of the poses. In recent years, some mainstream VR systems have released 6-degree-of-freedom (6-DoF) tracking devices providing positional and rotational information. Nevertheless, most solutions for reconstructing full-body poses rely on traditional inverse kinematics (IK) solutions, which often produce non-continuous and unnatural poses. In this article, we introduce SparsePoser, a novel deep learning-based solution for reconstructing a full-body pose from a reduced set of six tracking devices. Our system incorporates a convolutional-based autoencoder that synthesizes high-quality continuous human poses by learning the human motion manifold from motion capture data. Then, we employ a learned IK component, made of multiple lightweight feed-forward neural networks, to adjust the hands and feet toward the corresponding trackers. We extensively evaluate our method on publicly available motion capture datasets and with real-time live demos. We show that our method outperforms state-of-the-art techniques using IMU sensors or 6-DoF tracking devices, and can be used for users with different body dimensions and proportions.

Keypoint-Restored Inverse Kinematic Solver

HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recovery

HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation

Parametric Human Body Reconstruction Based on Sparse Key Points.

Mpose: Environment- and Subject-Agnostic 3D Skeleton Posture Reconstruction Leveraging a Single Mmwave Device

HierarIK - Hierarchical Inverse Kinematics Solver for Human Body and Hand Pose Estimation.

KinePose: A temporally optimized inverse kinematics technique for 6DOF human pose estimation with biomechanical constraints

Symmetry-aware Kinematic Skeleton Generation of a 3D Human Body Model.

Dynamic Human Body Reconstruction and Motion Tracking with Low-Cost Depth Cameras

Skeleton2Mesh - Kinematics Prior Injected Unsupervised Human Mesh Recovery.

3D Kinematics Estimation from Video with a Biomechanical Model and Synthetic Training Data

Fast 3D Human Body Reconstruction from Kinects

SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data

KITRO: Refining Human Mesh by 2D Clues and Kinematic-tree Rotation

Inverse kinematics using dynamic joint parameters: inverse kinematics animation synthesis learnt from sub-divided motion micro-segments

12-DOF analytic inverse kinematics solver for human motion control

3D Human Mesh Recovery with Sequentially Global Rotation Estimation

Reconstructing 3D human pose and shape from a single image and sparse IMUs

Motion Capture Research: 3D Human Pose Recovery Based on RGB Video Sequences

Markerless Motion Capture of Human Body Using PSO with Single Depth Camera

New multi-view human motion capture framework