Abstract:The multi-view 3D human pose estimation task relies on 2D human pose estimation for each view; however, severe occlusion, truncation, and human interaction lead to incorrect 2D human pose estimation for some views. The traditional "Matching-Lifting-Tracking" paradigm amplifies the incorrect 2D human pose into an incorrect 3D human pose, which significantly challenges the robustness of multi-view 3D human pose estimation. In this paper, we propose a novel method that tackles the inherent difficulties of the traditional paradigm. This method is rooted in the newly devised "Skeleton Pooling-Clustering-Tracking (SPCT)" paradigm. It initiates a 2D human pose estimation for each perspective. Then a symmetrical dilated network is created for skeleton pool estimation. Upon clustering the skeleton pool, we introduce and implement an innovative tracking method that is explicitly designed for the SPCT paradigm. The tracking method refines and filters the skeleton clusters, thereby enhancing the robustness of the multi-person 3D human pose estimation results. By coupling the skeleton pool with the tracking refinement process, our method obtains high-quality multi-person 3D human pose estimation results despite severe occlusions that produce erroneous 2D and 3D estimates. By employing the proposed SPCT paradigm and a computationally efficient network architecture, our method outperformed existing approaches regarding robustness on the Shelf, 4D Association, and CMU Panoptic datasets, and could be applied in practical scenarios such as markerless motion capture and animation production.

Simultaneous 3-D Human-Motion Tracking and Voxel Reconstruction

Model-based 3D human motion tracking and voxel reconstruction from sparse views

An accelerated human motion tracking system based on voxel reconstruction under complex environments

Marker-Less 3d Human Motion Capture With Monocular Image Sequence And Height-Maps

Digitize Your Body and Action in 3-D at Over 10 FPS: Real Time Dense Voxel Reconstruction and Marker-less Motion Tracking via GPU Acceleration.

A Two-Stage Scheme for Multi-View Human Pose Estimation

Dynamic Human Body Reconstruction and Motion Tracking with Low-Cost Depth Cameras

Reconstruction of 3D Human Motion Using Multiple View L1 Trackers

Two-camera-based Human Motion Capture

A Bayesian framework for 3D human motion tracking from monocular image

Efficient 3D Recovery of Human Motion in Monocular Video

Dynamic Multi-Person Mesh Recovery From Uncalibrated Multi-View Cameras

Towards robust 3d reconstruction of human motion from monocular video

Multiview 3-D Human Motion Tracking with Information Fusion Sampling

Accurate realtime full-body motion capture using a single depth camera

Human Motion Tracking with Less Constraint of Initial Posture from a Single RGB-D Sensor

Physics-Guided Human Motion Capture with Pose Probability Modeling

Skeleton Cluster Tracking for robust multi-view multi-person 3D human pose estimation

Monocular 3D human pose estimation via sequential second order cone programming

Imocap: Motion Capture from Internet Videos

Decoupling Human and Camera Motion from Videos in the Wild