Abstract:Training an accurate 3D human pose estimator often requires a large amount of 3D ground-truth data which is inefficient and costly to collect. Previous methods have either resorted to weakly supervised methods to reduce the demand of ground-truth data for training, or using synthetically-generated but photo-realistic samples to enlarge the training data pool. Nevertheless, the former methods mainly require either additional supervision, such as unpaired 3D ground-truth data, or the camera parameters in multiview settings. On the other hand, the latter methods require accurately textured models, illumination configurations and background which need careful engineering. To address these problems, we propose a domain adaptation framework with unsupervised knowledge transfer, which aims at leveraging the knowledge in multi-modality data of the easy-to-get synthetic depth datasets to better train a pose estimator on the real-world datasets. Specifically, the framework first trains two pose estimators on synthetically-generated depth images and human body segmentation masks with full supervision, while jointly learning a human body segmentation module from the predicted 2D poses. Subsequently, the learned pose estimator and the segmentation module are applied to the real-world dataset to unsupervisedly learn a new RGB image based 2D/3D human pose estimator. Here, the knowledge encoded in the supervised learning modules are used to regularize a pose estimator without ground-truth annotations. Comprehensive experiments demonstrate significant improvements over weakly supervised methods when no ground-truth annotations are available. Further experiments with ground-truth annotations show that the proposed framework can outperform state-of-the-art fully supervised methods. In addition, we conducted ablation studies to examine the impact of each loss term, as well as with different amount of supervisions signal.

Distill Knowledge from NRSfM for Weakly Supervised 3D Pose Learning

Learning Stereopsis from Geometric Synthesis for 6D Object Pose Estimation

Unsupervised Domain Adaptation for 3D Human Pose Estimation

Deductive Learning for Weakly-Supervised 3D Human Pose Estimation Via Uncalibrated Cameras.

Learning with Privileged Stereo Knowledge for Monocular Absolute 3d Human Pose Estimation

PoseNet3D: Learning Temporally Consistent 3D Human Pose via Knowledge Distillation

Geometry-Driven Self-Supervised Method for 3D Human Pose Estimation

Weakly-supervised Pre-training for 3D Human Pose Estimation via Perspective Knowledge

Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling

Region Deformer Networks for Unsupervised Depth Estimation from Unconstrained Monocular Videos

Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach

Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flows

Weakly Supervised 3D Multi-Person Pose Estimation for Large-Scale Scenes Based on Monocular Camera and Single LiDAR

Weakly-Supervised 3D Hand Reconstruction with Knowledge Prior and Uncertainty Guidance

Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimation

Self-Supervised Learning of Depth and Ego-Motion from Videos by Alternative Training and Geometric Constraints from 3-D to 2-D

X as Supervision: Contending with Depth Ambiguity in Unsupervised Monocular 3D Pose Estimation

Self-supervised Method for 3D Human Pose Estimation with Consistent Shape and Viewpoint Factorization.

Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation

NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimation

3D Hierarchical Refinement and Augmentation for Unsupervised Learning of Depth and Pose From Monocular Video