Abstract:Facial expression generation is one of the most challenging and long-sought aspects of character animation, with many interesting applications. The challenging task, traditionally having relied heavily on digital craftspersons, remains yet to be explored. In this paper, we introduce a generative framework for generating 3D facial expression sequences (i.e. 4D faces) that can be conditioned on different inputs to animate an arbitrary 3D face mesh. It is composed of two tasks: (1) Learning the generative model that is trained over a set of 3D landmark sequences, and (2) Generating 3D mesh sequences of an input facial mesh driven by the generated landmark sequences. The generative model is based on a Denoising Diffusion Probabilistic Model (DDPM), which has achieved remarkable success in generative tasks of other domains. While it can be trained unconditionally, its reverse process can still be conditioned by various condition signals. This allows us to efficiently develop several downstream tasks involving various conditional generation, by using expression labels, text, partial sequences, or simply a facial geometry. To obtain the full mesh deformation, we then develop a landmark-guided encoder-decoder to apply the geometrical deformation embedded in landmarks on a given facial mesh. Experiments show that our model has learned to generate realistic, quality expressions solely from the dataset of relatively small size, improving over the state-of-the-art methods. Videos and qualitative comparisons with other methods can be found at https://github.com/ZOUKaifeng/4DFM. Code and models will be made available upon acceptance.

3D Facial Expression Editing Based on the Dynamic Graph Model

Interactive Modeling of 3D Facial Expressions with Hierarchical Gaussian Process Latent Variable Models

Modeling Expressions of Peking Opera Facial Make-Ups

Modeling and Editing of Peking Opera Facial Makeup Expressions

GPU-Accelerated Parallel Realistic 3D Facial Expression Synthesis

Facial Expression Animation Based on Physical Model

Video Tracked Facial Expression Animation

Expressive 3D face synthesis by multi-space modeling

A Facial Expression Model For Human-Like Agent

4D Facial Expression Diffusion Model

A Novel Dynamic Model Capturing Spatial and Temporal Patterns for Facial Expression Analysis

Facial expression editing in video using a temporally-smooth factorization

Capturing Complex Spatio-temporal Relations among Facial Muscles for Facial Expression Recognition

Video-driven state-aware facial animation

Editing and Synthesis of 3D Facial Animation by Motion Propagation and Isomap Analysis

Quick Customization of Particular Human Face and Muscle-Driven Expression Animation

An experimental facial synthesis system using graph cut and gradient domain fusion

Synthesis of Combined Facial Expressions Using Anatomy-based Model

Shape and Expression Space of Realistic Human Faces

Personalized 3-D facial expression synthesis based on landmark constraint.

Sparse coding for flexible, robust 3D facial-expression synthesis.