Abstract:Training an accurate 3D human pose estimator often requires a large amount of 3D ground-truth data which is inefficient and costly to collect. Previous methods have either resorted to weakly supervised methods to reduce the demand of ground-truth data for training, or using synthetically-generated but photo-realistic samples to enlarge the training data pool. Nevertheless, the former methods mainly require either additional supervision, such as unpaired 3D ground-truth data, or the camera parameters in multiview settings. On the other hand, the latter methods require accurately textured models, illumination configurations and background which need careful engineering. To address these problems, we propose a domain adaptation framework with unsupervised knowledge transfer, which aims at leveraging the knowledge in multi-modality data of the easy-to-get synthetic depth datasets to better train a pose estimator on the real-world datasets. Specifically, the framework first trains two pose estimators on synthetically-generated depth images and human body segmentation masks with full supervision, while jointly learning a human body segmentation module from the predicted 2D poses. Subsequently, the learned pose estimator and the segmentation module are applied to the real-world dataset to unsupervisedly learn a new RGB image based 2D/3D human pose estimator. Here, the knowledge encoded in the supervised learning modules are used to regularize a pose estimator without ground-truth annotations. Comprehensive experiments demonstrate significant improvements over weakly supervised methods when no ground-truth annotations are available. Further experiments with ground-truth annotations show that the proposed framework can outperform state-of-the-art fully supervised methods. In addition, we conducted ablation studies to examine the impact of each loss term, as well as with different amount of supervisions signal.

Learning 3D Human Pose and Shape Estimation Using Uncertainty-Aware Body Part Segmentation

Unsupervised Domain Adaptation for 3D Human Pose Estimation

Single Image based 3D Human Pose Estimation via Uncertainty Learning

Unsupervised Universal Hierarchical Multi-Person 3D Pose Estimation for Natural Scenes

Utilizing Uncertainty in 2D Pose Detectors for Probabilistic 3D Human Mesh Recovery

High-precision Human Body Acquisition Via Multi-View Binocular Stereopsis

Robust Human Body Segmentation Based on Part Appearance and Spatial Constraint

Monocular Human Pose and Shape Reconstruction Using Part Differentiable Rendering

Mask as Supervision: Leveraging Unified Mask Information for Unsupervised 3D Pose Estimation

UPose3D: Uncertainty-Aware 3D Human Pose Estimation with Cross-View and Temporal Cues

Weakly Supervised Body Part Segmentation with Pose based Part Priors

Parametric Human Shape Reconstruction Via Bidirectional Silhouette Guidance

Probabilistic Estimation of 3D Human Shape and Pose with a Semantic Local Parametric Model

USR: Unsupervised Separated 3D Garment and Human Reconstruction via Geometry and Semantic Consistency

3D Human Pose and Shape Estimation with Dense Correspondence from a Single Depth Image

Sequential 3D Human Pose and Shape Estimation from Point Clouds

Self-supervised 3D Human Mesh Recovery from Noisy Point Clouds

Uncertainty-Aware Testing-Time Optimization for 3D Human Pose Estimation

Detailed Human Shape Estimation From A Single Image By Hierarchical Mesh Deformation

HEMlets PoSh: Learning Part-Centric Heatmap Triplets for 3D Human Pose and Shape Estimation

Embodied Uncertainty-Aware Object Segmentation