Abstract:Emerging Metaverse applications demand accessible, accurate, and easy-to-use tools for 3D digital human creations in order to depict different cultures and societies as if in the physical world. Recent large-scale vision-language advances pave the way to for novices to conveniently customize 3D content. However, the generated CG-friendly assets still cannot represent the desired facial traits for human characteristics. In this paper, we present DreamFace, a progressive scheme to generate personalized 3D faces under text guidance. It enables layman users to naturally customize 3D facial assets that are compatible with CG pipelines, with desired shapes, textures, and fine-grained animation capabilities. From a text input to describe the facial traits, we first introduce a coarse-to-fine scheme to generate the neutral facial geometry with a unified topology. We employ a selection strategy in the CLIP embedding space, and subsequently optimize both the details displacements and normals using Score Distillation Sampling from generic Latent Diffusion Model. Then, for neutral appearance generation, we introduce a dual-path mechanism, which combines the generic LDM with a novel texture LDM to ensure both the diversity and textural specification in the UV space. We also employ a two-stage optimization to perform SDS in both the latent and image spaces to significantly provides compact priors for fine-grained synthesis. Our generated neutral assets naturally support blendshapes-based facial animations. We further improve the animation ability with personalized deformation characteristics by learning the universal expression prior using the cross-identity hypernetwork. Notably, DreamFace can generate of realistic 3D facial assets with physically-based rendering quality and rich animation ability from video footage, even for fashion icons or exotic characters in cartoons and fiction movies.

FPGA Based Assembling of Facial Components for Human Face Construction

Face Synthesis (FASY) System for Generation of a Face Image from Human Description

Design and Implementation of Visual Processing System Based on FPGA

A synthesis method for personalized 3D face reconstruction

GPU-Accelerated Parallel Realistic 3D Facial Expression Synthesis

FPGA Implementation of Non-Parametric Stereo Matching Algorithm

An experimental facial synthesis system using graph cut and gradient domain fusion

Fuzzy Classification of Facial Component Parameters

Realistic Reconstruction of Human Face Based on Images

FPGA Implementation of Feature Detection Algorithm Based on High Level Synthesis

Fast face modeling based on features and flexible mesh model

Real-time face detection and lip feature extraction using field-programmable gate arrays

DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance

A 700fps Optimized Coarse-to-fine Shape Searching Based Hardware Accelerator for Face Alignment

Realistic 3D Face Modeling by Fusing Multiple 2D Images

Domain Engineering for Applied Monocular Reconstruction of Parametric Faces

Applied monocular reconstruction of parametric faces with domain engineering

Face Synthesis from Visual Attributes via Sketch using Conditional VAEs and GANs

Automatic Face Modeling and Synthesis Based on Image Pairs

SPECIFIC FACE SYNTHESIS BASED ON THE MODIFICATION OF THE GENERAL FACE MODEL

Fpga-based SoC design for real-time facial point detection using deep convolutional neural networks with dynamic partial reconfiguration