Abstract:Emerging Metaverse applications demand accessible, accurate, and easy-to-use tools for 3D digital human creations in order to depict different cultures and societies as if in the physical world. Recent large-scale vision-language advances pave the way to for novices to conveniently customize 3D content. However, the generated CG-friendly assets still cannot represent the desired facial traits for human characteristics. In this paper, we present DreamFace, a progressive scheme to generate personalized 3D faces under text guidance. It enables layman users to naturally customize 3D facial assets that are compatible with CG pipelines, with desired shapes, textures, and fine-grained animation capabilities. From a text input to describe the facial traits, we first introduce a coarse-to-fine scheme to generate the neutral facial geometry with a unified topology. We employ a selection strategy in the CLIP embedding space, and subsequently optimize both the details displacements and normals using Score Distillation Sampling from generic Latent Diffusion Model. Then, for neutral appearance generation, we introduce a dual-path mechanism, which combines the generic LDM with a novel texture LDM to ensure both the diversity and textural specification in the UV space. We also employ a two-stage optimization to perform SDS in both the latent and image spaces to significantly provides compact priors for fine-grained synthesis. Our generated neutral assets naturally support blendshapes-based facial animations. We further improve the animation ability with personalized deformation characteristics by learning the universal expression prior using the cross-identity hypernetwork. Notably, DreamFace can generate of realistic 3D facial assets with physically-based rendering quality and rich animation ability from video footage, even for fashion icons or exotic characters in cartoons and fiction movies.

Text2AC: A Framework for Game-Ready 2D Agent Character(AC) Generation from Natural Language

Make-A-Character: High Quality Text-to-3D Character Generation within Minutes

Text-driven Visual Prosody Generation for Embodied Conversational Agents

Zero-Shot Text-to-Parameter Translation for Game Character Auto-Creation

2D animation comic character action generation technology based on biomechanics simulation and artificial intelligence

Text2Avatar: Text to 3D Human Avatar Generation with Codebook-Driven Body Controllable Attribute

GenCA: A Text-conditioned Generative Model for Realistic and Drivable Codec Avatars

Driving Generative Agents With Their Personality

Digital Life Project: Autonomous 3D Characters with Social Intelligence

Crafting Customisable Characters with LLMs: Introducing SimsChat, a Persona-Driven Role-Playing Agent Framework

Generative AI Processes for 2D Platformer Game Character Design and Animation

AvatarStudio: High-fidelity and Animatable 3D Avatar Creation from Text

Text to Avatar in Multi-modal Human Computer Interface

DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance

TADA! Text to Animatable Digital Avatars

AgentAvatar: Disentangling Planning, Driving and Rendering for Photorealistic Avatar Agents

Fast and Robust Face-to-Parameter Translation for Game Character Auto-Creation

A Multimodal Approach of Generating 3D Human-Like Talking Agent.

HeadEvolver: Text to Head Avatars via Expressive and Attribute-Preserving Mesh Deformation

AI-generated characters for supporting personalized learning and well-being