Abstract:Training an accurate 3D human pose estimator often requires a large amount of 3D ground-truth data which is inefficient and costly to collect. Previous methods have either resorted to weakly supervised methods to reduce the demand of ground-truth data for training, or using synthetically-generated but photo-realistic samples to enlarge the training data pool. Nevertheless, the former methods mainly require either additional supervision, such as unpaired 3D ground-truth data, or the camera parameters in multiview settings. On the other hand, the latter methods require accurately textured models, illumination configurations and background which need careful engineering. To address these problems, we propose a domain adaptation framework with unsupervised knowledge transfer, which aims at leveraging the knowledge in multi-modality data of the easy-to-get synthetic depth datasets to better train a pose estimator on the real-world datasets. Specifically, the framework first trains two pose estimators on synthetically-generated depth images and human body segmentation masks with full supervision, while jointly learning a human body segmentation module from the predicted 2D poses. Subsequently, the learned pose estimator and the segmentation module are applied to the real-world dataset to unsupervisedly learn a new RGB image based 2D/3D human pose estimator. Here, the knowledge encoded in the supervised learning modules are used to regularize a pose estimator without ground-truth annotations. Comprehensive experiments demonstrate significant improvements over weakly supervised methods when no ground-truth annotations are available. Further experiments with ground-truth annotations show that the proposed framework can outperform state-of-the-art fully supervised methods. In addition, we conducted ablation studies to examine the impact of each loss term, as well as with different amount of supervisions signal.

Sim-to-Real Domain Adaptation for Deformation Classification

Unsupervised Domain Adaptation Approach for Vision-Based Semantic Understanding of Bridge Inspection Scenes Without Manual Annotations

Category-Level 6-D Object Pose Estimation With Shape Deformation for Robotic Grasp Detection

Unsupervised Domain Adaptation for 3D Human Pose Estimation

Domain Adaptation of Deformable Part-Based Models

Synthetic Data with Global-Local Level for Adversarial-based Domain Adaptation

Domain-Adaptive Single-View 3D Reconstruction

Domain Adaptation for Real-World Single View 3D Reconstruction

CAD Models to Real-World Images: A Practical Approach to Unsupervised Domain Adaptation in Industrial Object Classification

Learning from Synthetic Animals

From Virtual to Real World Visual Perception using Domain Adaptation -- The DPM as Example

CTS: Sim-to-Real Unsupervised Domain Adaptation on 3D Detection

Domain Adaptation for Head Pose Estimation Using Relative Pose Consistency

Visual Deformation Detection Using Soft Material Simulation for Pre-training of Condition Assessment Models

Multi-level joint distributed alignment-based domain adaptation for cross-scenario strip defect recognition

Controlling Human Shape and Pose in Text-to-Image Diffusion Models via Domain Adaptation

Unsupervised Domain Adaptation: from Simulation Engine to the RealWorld

Towards improving synthetic-to-real image correlation for instance recognition in structure monitoring

Increasing the Robustness of Deep Learning Models for Object Segmentation: A Framework for Blending Automatically Annotated Real and Synthetic Data.

Synthetic training data generation for deep learning based quality inspection

Towards Minimizing the LiDAR Sim-to-Real Domain Shift: Object-Level Local Domain Adaptation for 3D Point Clouds of Autonomous Vehicles