Abstract:Depth sensor based 3D human motion estimation hardware such as Kinect has made interactive applications more popular recently. However, it is still challenging to accurately recognize postures from a single depth camera due to the inherently noisy data derived from depth images and self-occluding action performed by the user. In this paper, we propose a new real-time probabilistic framework to enhance the accuracy of live captured postures that belong to one of the action classes in the database. We adopt the Gaussian Process model as a prior to leverage the position data obtained from Kinect and marker-based motion capture system. We also incorporate a temporal consistency term into the optimization framework to constrain the velocity variations between successive frames. To ensure that the reconstructed posture resembles the accurate parts of the observed posture, we embed a set of joint reliability measurements into the optimization framework. A major drawback of Gaussian Process is its cubic learning complexity when dealing with a large database due to the inverse of a covariance matrix. To solve the problem, we propose a new method based on a local mixture of Gaussian Processes, in which Gaussian Processes are defined in local regions of the state space. Due to the significantly decreased sample size in each local Gaussian Process, the learning time is greatly reduced. At the same time, the prediction speed is enhanced as the weighted mean prediction for a given sample is determined by the nearby local models only. Our system also allows incrementally updating a specific local Gaussian Process in real time, which enhances the likelihood of adapting to run-time postures that are different from those in the database. Experimental results demonstrate that our system can generate high quality postures even under severe self-occlusion situations, which is beneficial for real-time applications such as motion-based gaming and sport training.

Joint gait-pose manifold for video-based human motion estimation

Exploring Severe Occlusion: Multi-Person 3D Pose Estimation with Gated Convolution.

Gait analysis of human locomotion based on motion capture system

Joint-Motion Mutual Learning for Pose Estimation in Videos

Human Identification Based on Gait Manifold

Human motion tracking by temporal-spatial local gaussian process experts

Kinect Posture Reconstruction Based on a Local Mixture of Gaussian Process Models

MoManifold: Learning to Measure 3D Human Motion via Decoupled Joint Acceleration Manifolds

Gaussian Particle Filter Based Pose and Motion Estimation

Personalized motion kernel learning for human pose estimation

JointsGait:A model-based Gait Recognition Method based on Gait Graph Convolutional Networks and Joints Relationship Pyramid Mapping

Towards Complete-View and High-Level Pose-based Gait Recognition

Kinematics Modeling Network for Video-based Human Pose Estimation

MLP-JCG: Multi-Layer Perceptron with Joint-Coordinate Gating for Efficient 3D Human Pose Estimation

Learning Dynamical Human-Joint Affinity for 3D Pose Estimation in Videos

Monocular Tracking 3D People by Gaussian Process Spatio-Temporal Variable Model

Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modeling

SPGformer: Serial–Parallel Hybrid GCN-Transformer With Graph-Oriented Encoder for 2-D-to-3-D Human Pose Estimation

ManiPose: Manifold-Constrained Multi-Hypothesis 3D Human Pose Estimation

Whole-Body Joint Angle Estimation for Real-Time Humanoid Robot Imitation Based on Gaussian Process Dynamical Model and Particle Filter