Design2Cloth: 3D Cloth Generation from 2D Masks

Jiali Zheng,Rolandos Alexandros Potamias,Stefanos Zafeiriou

2024-04-03

Abstract:In recent years, there has been a significant shift in the field of digital avatar research, towards modeling, animating and reconstructing clothed human representations, as a key step towards creating realistic avatars. However, current 3D cloth generation methods are garment specific or trained completely on synthetic data, hence lacking fine details and realism. In this work, we make a step towards automatic realistic garment design and propose Design2Cloth, a high fidelity 3D generative model trained on a real world dataset from more than 2000 subject scans. To provide vital contribution to the fashion industry, we developed a user-friendly adversarial model capable of generating diverse and detailed clothes simply by drawing a 2D cloth mask. Under a series of both qualitative and quantitative experiments, we showcase that Design2Cloth outperforms current state-of-the-art cloth generative models by a large margin. In addition to the generative properties of our network, we showcase that the proposed method can be used to achieve high quality reconstructions from single in-the-wild images and 3D scans. Dataset, code and pre-trained model will become publicly available.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

This paper aims to address the problem of 3D clothing generation, specifically the generation of high-fidelity 3D clothing models from simple 2D masks. Current methods either focus on specific garments or heavily rely on synthetic data, lacking details and realism. To tackle this, the research team created a high-fidelity 3D generation model called Design2Cloth, trained on a large-scale real-world 3D clothing dataset consisting of over 2000 individual garments, capable of generating diverse and detailed clothing.

Design2Cloth introduces a user-friendly adversarial model where users only need to draw 2D clothing masks to generate the clothing, simplifying the generation process. Furthermore, since the model is differentiable, it can be used for high-quality reconstruction from a single wild image and 3D scans. The paper demonstrates that Design2Cloth significantly outperforms existing state-of-the-art methods in clothing generation and reconstruction tasks, and provides public access links to relevant datasets, code, and pretrained models.