Abstract:We introduce TADA, a simple-yet-effective approach that takes textual descriptions and produces expressive 3D avatars with high-quality geometry and lifelike textures, that can be animated and rendered with traditional graphics pipelines. Existing text-based character generation methods are limited in terms of geometry and texture quality, and cannot be realistically animated due to inconsistent alignment between the geometry and the texture, particularly in the face region. To overcome these limitations, TADA leverages the synergy of a 2D diffusion model and an animatable parametric body model. Specifically, we derive an optimizable high-resolution body model from SMPL-X with 3D displacements and a texture map, and use hierarchical rendering with score distillation sampling (SDS) to create high-quality, detailed, holistic 3D avatars from text. To ensure alignment between the geometry and texture, we render normals and RGB images of the generated character and exploit their latent embeddings in the SDS training process. We further introduce various expression parameters to deform the generated character during training, ensuring that the semantics of our generated character remain consistent with the original SMPL-X model, resulting in an animatable character. Comprehensive evaluations demonstrate that TADA significantly surpasses existing approaches on both qualitative and quantitative measures. TADA enables creation of large-scale digital character assets that are ready for animation and rendering, while also being easily editable through natural language. The code will be public for research purposes.

ClipMatrix: Text-controlled Creation of 3D Textured Meshes

CLIP-Mesh: Generating textured meshes from text using pretrained image-text models

Text2Mesh: Text-Driven Neural Stylization for Meshes

Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Models

TEXTure: Text-Guided Texturing of 3D Shapes

TextDeformer: Geometry Manipulation using Text Guidance

CLIP-Actor: Text-Driven Recommendation and Stylization for Animating Human Meshes

3D Creation at Your Fingertips: From Text or Image to 3D Assets

Meta 3D Gen

HeadSculpt: Crafting 3D Head Avatars with Text

Control3D: Towards Controllable Text-to-3D Generation

TextMesh: Generation of Realistic 3D Meshes From Text Prompts

Text-guided Controllable Mesh Refinement for Interactive 3D Modeling

AvatarCLIP: zero-shot text-driven generation and animation of 3D avatars

Text-Free Controllable 3D Point Cloud Generation

AvatarStudio: High-fidelity and Animatable 3D Avatar Creation from Text

HeadEvolver: Text to Head Avatars via Expressive and Attribute-Preserving Mesh Deformation

Text-Free Controllable 3-D Point Cloud Generation

TADA! Text to Animatable Digital Avatars

ControlDreamer: Blending Geometry and Style in Text-to-3D

MagicClay: Sculpting Meshes With Generative Neural Fields