4D-DRESS: A 4D Dataset of Real-world Human Clothing with Semantic Annotations

Wenbo Wang,Hsuan-I Ho,Chen Guo,Boxiang Rong,Artur Grigorev,Jie Song,Juan Jose Zarate,Otmar Hilliges

2024-04-29

Abstract:The studies of human clothing for digital avatars have predominantly relied on synthetic datasets. While easy to collect, synthetic data often fall short in realism and fail to capture authentic clothing dynamics. Addressing this gap, we introduce 4D-DRESS, the first real-world 4D dataset advancing human clothing research with its high-quality 4D textured scans and garment meshes. 4D-DRESS captures 64 outfits in 520 human motion sequences, amounting to 78k textured scans. Creating a real-world clothing dataset is challenging, particularly in annotating and segmenting the extensive and complex 4D human scans. To address this, we develop a semi-automatic 4D human parsing pipeline. We efficiently combine a human-in-the-loop process with automation to accurately label 4D scans in diverse garments and body movements. Leveraging precise annotations and high-quality garment meshes, we establish several benchmarks for clothing simulation and reconstruction. 4D-DRESS offers realistic and challenging data that complements synthetic sources, paving the way for advancements in research of lifelike human clothing. Website:

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The paper aims to address the issues of realism and dynamic simulation in the study of human clothing in digital avatars. Specifically, existing synthetic datasets, while easy to collect and large in quantity, lack realism and dynamic clothing aspects. To bridge this gap, the authors introduce the 4D-DRESS dataset, the first real-world 4D clothing dataset, which includes high-quality 4D texture scans and clothing meshes.

The features of the 4D-DRESS dataset are as follows:
- It includes 64 different outfits, capturing 78,000 frames of high-resolution 4D scans in over 520 action sequences.
- Provides vertex-level semantic labels and includes the corresponding SMPL(-X) body models.
- The dataset covers various types of clothing, including tops, bottoms, outerwear, and complex loose garments such as skirts and jackets.
- By developing a semi-automatic 4D human parsing pipeline, it addresses the annotation and segmentation issues of large-scale 4D scans.

The main contributions of the paper include:
1. **The first real-world 4D clothing dataset**: Provides high-precision 4D texture scans, vertex-level semantic labels, clothing meshes, and corresponding parametric body models.
2. **Semi-automatic 4D human parsing pipeline**: Efficiently annotates the data.
3. **Benchmarking**: Demonstrates the dataset's utility in clothing simulation and reconstruction tasks, proving that existing algorithms struggle to handle complex real-world clothing.

Overall, the 4D-DRESS dataset aims to provide a more realistic and challenging benchmarking platform for human clothing research in the fields of computer vision and graphics.