Abstract:Traditional 3D garment creation is labor-intensive, involving sketching, modeling, UV mapping, and texturing, which are time-consuming and costly. Recent advances in diffusion-based generative models have enabled new possibilities for 3D garment generation from text prompts, images, and videos. However, existing methods either suffer from inconsistencies among multi-view images or require additional processes to separate cloth from the underlying human model. In this paper, we propose GarmentDreamer, a novel method that leverages 3D Gaussian Splatting (GS) as guidance to generate wearable, simulation-ready 3D garment meshes from text prompts. In contrast to using multi-view images directly predicted by generative models as guidance, our 3DGS guidance ensures consistent optimization in both garment deformation and texture synthesis. Our method introduces a novel garment augmentation module, guided by normal and RGBA information, and employs implicit Neural Texture Fields (NeTF) combined with Score Distillation Sampling (SDS) to generate diverse geometric and texture details. We validate the effectiveness of our approach through comprehensive qualitative and quantitative experiments, showcasing the superior performance of GarmentDreamer over state-of-the-art alternatives. Our project page is available at:

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to automatically generate high - quality, simulatable 3D clothing mesh models with diverse geometric and texture details given a text prompt. Traditional 3D clothing creation methods require a great deal of manual labor, including steps such as sketching, modeling, UV mapping, and texturing, and these processes are time - consuming and costly. Although generative techniques based on diffusion models have made progress in generating 3D clothing from text, images, and videos, existing methods either have inconsistencies between multi - view images or require additional processing steps to separate the clothing from the human model. Therefore, generating 3D clothing that can be directly used for simulation with high - fidelity details remains a challenge. **Specifically, the main objectives of the paper include**: 1. **Generate high - quality 3D clothing models**: By using 3D Gaussian Splatting (3DGS) as guidance, generate wearable, simulation - ready 3D clothing mesh models that can be directly generated from text prompts. 2. **Solve the multi - view inconsistency problem**: Different from directly using multi - view images predicted by the generative model as guidance, 3DGS guidance ensures multi - view consistency in clothing deformation and texture synthesis. 3. **Introduce a novel clothing enhancement module**: This module uses normal and RGBA information for guidance and combines Implicit Neural Texture Fields (NeTF) and Variational Score Distillation (VSD) techniques to generate diverse geometric and texture details. 4. **Optimize texture quality**: By optimizing the Implicit Neural Texture Fields and using VSD for fine - tuning, generate high - quality clothing textures. **The main contributions of the paper include**: - Propose a new 3D clothing synthesis method that uses diffusion models and 3DGS as references to generate simulation - ready 3D clothing from text prompts. - Design a new clothing deformation module that uses normal and RGBA information to generate diverse, high - quality clothing with complex geometric details in the coarse - to - fine optimization stage. - Demonstrate how to reconstruct and fine - tune Implicit Neural Texture Fields through VSD loss to generate high - quality clothing textures. - Through comprehensive qualitative and quantitative experiments, verify the superior performance of GarmentDreamer compared to existing methods. In summary, this paper aims to solve key problems in 3D clothing generation by combining advanced generative models and 3D guidance techniques, thereby achieving the automatic generation of high - quality, simulatable 3D clothing from text prompts.

GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Details

ClotheDreamer: Text-Guided Garment Generation with 3D Gaussians

An Integrated Method of 3D Garment Design

Garment3DGen: 3D Garment Stylization and Texture Generation

Design 3D Garments for Scanned Human Bodies

DressCode: Autoregressively Sewing and Generating Garments from Text Guidance

New Fashion: Personalized 3D Design with a Single Sketch Input

DreamFit: Garment-Centric Human Generation via a Lightweight Anything-Dressing Encoder

AnyDressing: Customizable Multi-Garment Virtual Dressing via Latent Diffusion Models

Motion Guided Deep Dynamic 3D Garments

StableGarment: Garment-Centric Generation via Stable Diffusion

Garment4D: Garment Reconstruction from Point Cloud Sequences

A survey on CAD methods in 3D garment design

GSNet: Generating 3D Garment Animation Via Graph Skinning Network

Generating diverse clothed 3D human animations via a generative model

GAN-Based Garment Generation Using Sewing Pattern Images

WordRobe: Text-Guided Generation of Textured 3D Garments

Cloth2Tex: A Customized Cloth Texture Generation Pipeline for 3D Virtual Try-On

Gaussian Garments: Reconstructing Simulation-Ready Clothing with Photorealistic Appearance from Multi-View Video

Dynamic neural garments