DressCode: Autoregressively Sewing and Generating Garments from Text Guidance

Kai He,Kaixin Yao,Qixuan Zhang,Jingyi Yu,Lingjie Liu,Lan Xu

2024-06-15

Abstract:Apparel's significant role in human appearance underscores the importance of garment digitalization for digital human creation. Recent advances in 3D content creation are pivotal for digital human creation. Nonetheless, garment generation from text guidance is still nascent. We introduce a text-driven 3D garment generation framework, DressCode, which aims to democratize design for novices and offer immense potential in fashion design, virtual try-on, and digital human creation. We first introduce SewingGPT, a GPT-based architecture integrating cross-attention with text-conditioned embedding to generate sewing patterns with text guidance. We then tailor a pre-trained Stable Diffusion to generate tile-based Physically-based Rendering (PBR) textures for the garments. By leveraging a large language model, our framework generates CG-friendly garments through natural language interaction. It also facilitates pattern completion and texture editing, streamlining the design process through user-friendly interaction. This framework fosters innovation by allowing creators to freely experiment with designs and incorporate unique elements into their work. With comprehensive evaluations and comparisons with other state-of-the-art methods, our method showcases superior quality and alignment with input prompts. User studies further validate our high-quality rendering results, highlighting its practical utility and potential in production settings. Our project page is <a class="link-external link-https" href="https://IHe-KaiI.github.io/DressCode/" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition,Graphics

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that the current technology for text - based clothing generation is still in its infancy, especially lacking high - quality sewing patterns and physics - based texture generation capabilities. The author proposes a framework named DressCode, aiming to generate high - quality 3D clothing through natural - language interaction, including customized sewing patterns and physics - based rendering (PBR) textures. This framework can not only lower the design threshold for non - professionals, but also offer great potential in fields such as fashion design, virtual fitting, and digital human creation.

Specifically, the paper addresses the following key issues:

1. **High - quality sewing pattern generation**: Existing clothing generation methods can usually only generate meshes or neural fields, and these representations are incompatible with the digital clothing production workflow. DressCode generates high - quality sewing patterns by introducing SewingGPT, which utilizes the GPT architecture, and these patterns can be directly used for physical simulation and animation production.

2. **Physics - based texture generation**: Traditional 3D content generation methods are deficient in generating physics - based textures. DressCode generates high - quality PBR textures, including diffuse maps, normal maps, and roughness maps, by adjusting the pre - trained Stable Diffusion model, thereby improving the rendering quality.

3. **Natural - language interaction**: Most existing methods rely on complex software tools and are not user - friendly enough for non - professionals. DressCode allows users to generate and edit clothing through natural - language descriptions by integrating large - language models, greatly reducing the design threshold.

4. **Natural wrinkles of multi - layer clothing**: In a production environment, it is usually necessary to generate multiple clothing items simultaneously and wrinkle them naturally on a mannequin. DressCode achieves natural wrinkles of multi - layer clothing by using the sewing pattern representation, solving the limitations of mesh - or implicit - field - based methods in this regard.

5. **Pattern completion and texture editing**: DressCode not only supports generating complete sewing patterns from scratch, but can also complete them based on partial pattern information and text prompts. In addition, because the sewing pattern representation is used, the generated UV mapping has a structured property, which is convenient for texture editing at specific locations.

Through these innovations, DressCode significantly improves the quality and practicality of text - based clothing generation, providing new possibilities for digital clothing design and creation.