Abstract:The model-based estimation of 3D animal pose and shape from images enables computational modeling of animal behavior. Training models for this purpose requires large amounts of labeled image data with precise pose and shape annotations. However, capturing such data requires the use of multi-view or marker-based motion-capture systems, which are impractical to adapt to wild animals in situ and impossible to scale across a comprehensive set of animal species. Some have attempted to address the challenge of procuring training data by pseudo-labeling individual real-world images through manual 2D annotation, followed by 3D-parameter optimization to those labels. While this approach may produce silhouette-aligned samples, the obtained pose and shape parameters are often implausible due to the ill-posed nature of the monocular fitting problem. Sidestepping real-world ambiguity, others have designed complex synthetic-data-generation pipelines leveraging video-game engines and collections of artist-designed 3D assets. Such engines yield perfect ground-truth annotations but are often lacking in visual realism and require considerable manual effort to adapt to new species or environments. Motivated by these shortcomings, we propose an alternative approach to synthetic-data generation: rendering with a conditional image-generation model. We introduce a pipeline that samples a diverse set of poses and shapes for a variety of mammalian quadrupeds and generates realistic images with corresponding ground-truth pose and shape parameters. To demonstrate the scalability of our approach, we introduce GenZoo, a synthetic dataset containing one million images of distinct subjects. We train a 3D pose and shape regressor on GenZoo, which achieves state-of-the-art performance on a real-world animal pose and shape estimation benchmark, despite being trained solely on synthetic data. <a class="link-external link-https" href="https://genzoo.is.tue.mpg.de" rel="external noopener nofollow">this https URL</a>

Learning from Synthetic Animals

Learning Part Segmentation from Synthetic Animals

Prior-Aware Synthetic Data to the Rescue: Animal Pose Estimation with Very Limited Real Data

Pose Recognition in the Wild: Animal pose estimation using Agglomerative Clustering and Contrastive Learning

SuperAnimal pretrained pose estimation models for behavioral analysis

SPAC-Net: Synthetic Pose-aware Animal ControlNet for Enhanced Pose Estimation

Cross-Domain Adaptation for Animal Pose Estimation.

Learning the 3D Fauna of the Web

Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videos

ZebraPose: Zebra Detection and Pose Estimation using only Synthetic Data

A Horse with no Labels: Self-Supervised Horse Pose Estimation from Unlabelled Images and Synthetic Prior

Coarse-to-fine Animal Pose and Shape Estimation

Dessie: Disentanglement for Articulated 3D Horse Shape and Pose Estimation from Images

MagicPony: Learning Articulated 3D Animals in the Wild

Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts

Learning 3D Deformation of Animals from 2D Images

Learning From Synthetic Images via Active Pseudo-Labeling

Generative Zoo

Two-stage Synthetic Supervising and Multi-view Consistency Self-supervising based Animal 3D Reconstruction by Single Image

SyDog: A Synthetic Dog Dataset for Improved 2D Pose Estimation

Weakly and Semi Supervised Human Body Part Parsing Via Pose-Guided Knowledge Transfer