EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wild

Manuel Kaufmann,Jie Song,Chen Guo,Kaiyue Shen,Tianjian Jiang,Chengcheng Tang,Juan Zarate,Otmar Hilliges

DOI: https://doi.org/10.48550/arXiv.2308.16894

2023-09-01

Abstract:We present EMDB, the Electromagnetic Database of Global 3D Human Pose and Shape in the Wild. EMDB is a novel dataset that contains high-quality 3D SMPL pose and shape parameters with global body and camera trajectories for in-the-wild videos. We use body-worn, wireless electromagnetic (EM) sensors and a hand-held iPhone to record a total of 58 minutes of motion data, distributed over 81 indoor and outdoor sequences and 10 participants. Together with accurate body poses and shapes, we also provide global camera poses and body root trajectories. To construct EMDB, we propose a multi-stage optimization procedure, which first fits SMPL to the 6-DoF EM measurements and then refines the poses via image observations. To achieve high-quality results, we leverage a neural implicit avatar model to reconstruct detailed human surface geometry and appearance, which allows for improved alignment and smoothness via a dense pixel-level objective. Our evaluations, conducted with a multi-view volumetric capture system, indicate that EMDB has an expected accuracy of 2.3 cm positional and 10.6 degrees angular error, surpassing the accuracy of previous in-the-wild datasets. We evaluate existing state-of-the-art monocular RGB methods for camera-relative and global pose estimation on EMDB. EMDB is publicly available under <a class="link-external link-https" href="https://ait.ethz.ch/emdb" rel="external noopener nofollow">this https URL</a>

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to accurately estimate 3D human pose and shape, as well as the global trajectories of the camera and the human body in natural environments (i.e., unrestricted outdoor or indoor environments). Existing methods have limitations when dealing with global pose estimation in moving - camera scenarios, especially when occlusion, complex postures, and uncommon activities are involved. In addition, there is currently a lack of comprehensive datasets that can provide accurate 3D human pose, shape, and global trajectories, which restricts the development of related research. Therefore, this paper proposes a new dataset, EMDB (Electromagnetic Database of Global 3D Human Pose and Shape in the Wild), and a new method, EMP (Electromagnetic Poser), aiming to overcome these challenges and provide high - quality human pose and shape parameters, as well as global trajectory estimation.

Specifically, the main contributions of the paper include:

1. **EMDB Dataset**: This is the first 3D human pose and shape dataset recorded in unrestricted, moving natural environments, providing high - precision SMPL pose and shape parameters, as well as global trajectory estimation.
2. **EMP Method**: This is a multi - stage optimization method that fuses electromagnetic sensor measurement data, image data, and camera pose to generate high - precision SMPL parameters and global trajectory estimation.
3. **Evaluation**: The accuracy of the EMP method was evaluated in detail and compared with existing methods, demonstrating the high quality and accuracy of the EMDB dataset.

Through these contributions, the paper aims to promote research in the field of 3D human pose and shape estimation, especially in applications in natural environments.