Abstract:Depth sensor based 3D human motion estimation hardware such as Kinect has made interactive applications more popular recently. However, it is still challenging to accurately recognize postures from a single depth camera due to the inherently noisy data derived from depth images and self-occluding action performed by the user. In this paper, we propose a new real-time probabilistic framework to enhance the accuracy of live captured postures that belong to one of the action classes in the database. We adopt the Gaussian Process model as a prior to leverage the position data obtained from Kinect and marker-based motion capture system. We also incorporate a temporal consistency term into the optimization framework to constrain the velocity variations between successive frames. To ensure that the reconstructed posture resembles the accurate parts of the observed posture, we embed a set of joint reliability measurements into the optimization framework. A major drawback of Gaussian Process is its cubic learning complexity when dealing with a large database due to the inverse of a covariance matrix. To solve the problem, we propose a new method based on a local mixture of Gaussian Processes, in which Gaussian Processes are defined in local regions of the state space. Due to the significantly decreased sample size in each local Gaussian Process, the learning time is greatly reduced. At the same time, the prediction speed is enhanced as the weighted mean prediction for a given sample is determined by the nearby local models only. Our system also allows incrementally updating a specific local Gaussian Process in real time, which enhances the likelihood of adapting to run-time postures that are different from those in the database. Experimental results demonstrate that our system can generate high quality postures even under severe self-occlusion situations, which is beneficial for real-time applications such as motion-based gaming and sport training.

Learning Analytical Posterior Probability for Human Mesh Recovery

MH‐HMR: Human mesh recovery from monocular images via multi‐hypothesis learning

Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery

Visibility-Aware Human Mesh Recovery Via Balancing Dense Correspondence and Probability Model

Learning Human Mesh Recovery in 3D Scenes

HybrIK-X: Hybrid Analytical-Neural Inverse Kinematics for Whole-body Mesh Recovery

Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Pose

ARTS: Semi-Analytical Regressor using Disentangled Skeletal Representations for Human Mesh Recovery from Videos

Generative Approach for Probabilistic Human Mesh Recovery using Diffusion Models

Self-supervised 3D Human Mesh Recovery from Noisy Point Clouds

Skeleton2Mesh - Kinematics Prior Injected Unsupervised Human Mesh Recovery.

Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Views

3D Human Mesh Recovery with Sequentially Global Rotation Estimation

Learning Local Recurrent Models for Human Mesh Recovery

Dynamic Multi-Person Mesh Recovery From Uncalibrated Multi-View Cameras

Kinect Posture Reconstruction Based on a Local Mixture of Gaussian Process Models

Privacy preserving for AI-based 3D human pose recovery and retargeting

End-to-end Recovery of Human Shape and Pose

Deep learning for 3D human pose estimation and mesh recovery: A survey

Temporally Coherent Full 3D Mesh Human Pose Recovery from Monocular Video

PC-HMR: Pose Calibration for 3D Human Mesh Recovery from 2D Images/Videos