Abstract:Synthetic data is increasingly being used to address the lack of labeled images in uncommon domains for deep learning tasks. A prominent example is 2D pose estimation of animals, particularly wild species like zebras, for which collecting real-world data is complex and impractical. However, many approaches still require real images, consistency and style constraints, sophisticated animal models, and/or powerful pre-trained networks to bridge the syn-to-real gap. Moreover, they often assume that the animal can be reliably detected in images or videos, a hypothesis that often does not hold, e.g. in wildlife scenarios or aerial images. To solve this, we use synthetic data generated with a 3D photorealistic simulator to obtain the first synthetic dataset that can be used for both detection and 2D pose estimation of zebras without applying any of the aforementioned bridging strategies. Unlike previous works, we extensively train and benchmark our detection and 2D pose estimation models on multiple real-world and synthetic datasets using both pre-trained and non-pre-trained backbones. These experiments show how the models trained from scratch and only with synthetic data can consistently generalize to real-world images of zebras in both tasks. Moreover, we show it is possible to easily generalize those same models to 2D pose estimation of horses with a minimal amount of real-world images to account for the domain transfer. Code, results, trained models; and the synthetic, training, and validation data, including 104K manually labeled frames, are provided as open-source at <a class="link-external link-https" href="https://zebrapose.is.tue.mpg.de/" rel="external noopener nofollow">this https URL</a>

Improving 2D Human Pose Estimation in Rare Camera Views with Synthetic Data

MoCap-guided Data Augmentation for 3D Pose Estimation in the Wild

Image-Based Synthesis for Deep 3D Human Pose Estimation

Improvements in 3D Hand Pose Estimation Using Synthetic Data

Towards Generalization of 3D Human Pose Estimation In The Wild

Learning 3-D Human Pose Estimation from Catadioptric Videos

Human Pose Estimation in Monocular Omnidirectional Top-View Images

VirtualPose: Learning Generalizable 3D Human Pose Models from Virtual Data

A Survey on Monocular 3D Human Pose Estimation

Sim2real transfer learning for 3D human pose estimation: motion to the rescue

SimpleEgo: Predicting Probabilistic Body Pose from Egocentric Cameras

Overcoming Data Deficiency for Multi-Person Pose Estimation

SoloPose: One-Shot Kinematic 3D Human Pose Estimation with Video Data Augmentation

UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Model

D-PoSE: Depth as an Intermediate Representation for 3D Human Pose and Shape Estimation

Prior-Aware Synthetic Data to the Rescue: Animal Pose Estimation with Very Limited Real Data

xR-EgoPose: Egocentric 3D Human Pose from an HMD Camera

ZebraPose: Zebra Detection and Pose Estimation using only Synthetic Data

Resolving 3D Human Pose Ambiguities with 3D Scene Constraints

Robust Estimation of 3D Human Poses from a Single Image

Adapted Human Pose: Monocular 3D Human Pose Estimation with Zero Real 3D Pose Data