Abstract:We introduce a new dataset, Human3.6M, of 3.6 Million accurate 3D Human poses, acquired by recording the performance of 5 female and 6 male subjects, under 4 different viewpoints, for training realistic human sensing systems and for evaluating the next generation of human pose estimation models and algorithms. Besides increasing the size of the datasets in the current state-of-the-art by several orders of magnitude, we also aim to complement such datasets with a diverse set of motions and poses encountered as part of typical human activities (taking photos, talking on the phone, posing, greeting, eating, etc.), with additional synchronized image, human motion capture, and time of flight (depth) data, and with accurate 3D body scans of all the subject actors involved. We also provide controlled mixed reality evaluation scenarios where 3D human models are animated using motion capture and inserted using correct 3D geometry, in complex real environments, viewed with moving cameras, and under occlusion. Finally, we provide a set of large-scale statistical models and detailed evaluation baselines for the dataset illustrating its diversity and the scope for improvement by future work in the research community. Our experiments show that our best large-scale model can leverage our full training set to obtain a 20% improvement in performance compared to a training set of the scale of the largest existing public dataset for this problem. Yet the potential for improvement by leveraging higher capacity, more complex models with our large dataset, is substantially vaster and should stimulate future research. The dataset together with code for the associated large-scale learning models, features, visualization tools, as well as the evaluation server, is available online at http://vision.imar.ro/human3.6m.

SynBody: Synthetic Dataset with Layered Human Models for 3D Human Perception and Modeling

SMPLX-Lite: A Realistic and Drivable Avatar Benchmark with Rich Geometry and Texture Annotations

UltraPose: Synthesizing Dense Pose with 1 Billion Points by Human-body Decoupling 3D Model

BCNet: Learning Body and Cloth Shape from A Single Image

HuMMan: Multi-Modal 4D Human Dataset for Versatile Sensing and Modeling

SynPlay: Importing Real-world Diversity for a Synthetic Human Dataset

HSPACE: Synthetic Parametric Humans Animated in Complex Environments

CLOTH3D: Clothed 3D Humans

C3I-SynFace: A synthetic head pose and facial depth dataset using seed virtual human models.

Procedural Humans for Computer Vision

3D Human Reconstruction in the Wild with Synthetic Data Using Generative Models

Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humans

LatentHuman: Shape-and-Pose Disentangled Latent Representation for Human Bodies

Implicit Neural Representations With Structured Latent Codes for Human Body Modeling

Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments

Synthetic Data for Multi-Parameter Camera-Based Physiological Sensing

Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape

Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Dataset

4D-DRESS: A 4D Dataset of Real-world Human Clothing with Semantic Annotations

AnthroNet: Conditional Generation of Humans via Anthropometrics

3D Segmentation of Humans in Point Clouds with Synthetic Data