Abstract:Facial expression generation is one of the most challenging and long-sought aspects of character animation, with many interesting applications. The challenging task, traditionally having relied heavily on digital craftspersons, remains yet to be explored. In this paper, we introduce a generative framework for generating 3D facial expression sequences (i.e. 4D faces) that can be conditioned on different inputs to animate an arbitrary 3D face mesh. It is composed of two tasks: (1) Learning the generative model that is trained over a set of 3D landmark sequences, and (2) Generating 3D mesh sequences of an input facial mesh driven by the generated landmark sequences. The generative model is based on a Denoising Diffusion Probabilistic Model (DDPM), which has achieved remarkable success in generative tasks of other domains. While it can be trained unconditionally, its reverse process can still be conditioned by various condition signals. This allows us to efficiently develop several downstream tasks involving various conditional generation, by using expression labels, text, partial sequences, or simply a facial geometry. To obtain the full mesh deformation, we then develop a landmark-guided encoder-decoder to apply the geometrical deformation embedded in landmarks on a given facial mesh. Experiments show that our model has learned to generate realistic, quality expressions solely from the dataset of relatively small size, improving over the state-of-the-art methods. Videos and qualitative comparisons with other methods can be found at https://github.com/ZOUKaifeng/4DFM. Code and models will be made available upon acceptance.

Dynamic Facial Expression Synthesis Driven by Deformable Semantic Parts.

Expression Conditional Gan for Facial Expression-to-Expression Translation.

GPU-Accelerated Parallel Realistic 3D Facial Expression Synthesis

Audio-driven Talking Face Video Generation with Natural Head Pose

Video-driven state-aware facial animation

Geometry Guided Adversarial Facial Expression Synthesis

Expressive Face Animation Synthesis Based on Dynamic Mapping Method

Facial Landmarks and Expression Label Guided Photorealistic Facial Expression Synthesis

Data-driven facial expression synthesis via Laplacian deformation

Facial Expression Synthesis Based on Facial Expression Decomposition and Warping

Facial Expression Synthesis Based on Semantic Dimensions

Dynamic Facial Expression Generation on Hilbert Hypersphere With Conditional Wasserstein Generative Adversarial Nets

Image-to-Video Generation via 3D Facial Dynamics

Comp-GAN

High Resolution Acquisition, Learning And Transfer Of Dynamic 3-D Facial Expressions

A data-driven approach for facial expression synthesis in video

Active appearance model-based expression synthesis method

Realistic Dynamic Facial Textures from a Single Image Using GANs

4D Facial Expression Diffusion Model

Emotional Conversation: Empowering Talking Faces with Cohesive Expression, Gaze and Pose Generation

Expressive facial animation synthesis by learning speech coarticulation and expression spaces