Abstract:We present a versatile model, FaceAnime, for various video generation tasks from still images. Video generation from a single face image is an interesting problem and usually tackled by utilizing Generative Adversarial Networks (GANs) to integrate information from the input face image and a sequence of sparse facial landmarks. However, the generated face images usually suffer from quality loss, image distortion, identity change, and expression mismatching due to the weak representation capacity of the facial landmarks. In this paper, we propose to "imagine" a face video from a single face image according to the reconstructed 3D face dynamics, aiming to generate a realistic and identity-preserving face video, with precisely predicted pose and facial expression. The 3D dynamics reveal changes of the facial expression and motion, and can serve as a strong prior knowledge for guiding highly realistic face video generation. In particular, we explore face video prediction and exploit a well-designed 3D dynamic prediction network to predict a 3D dynamic sequence for a single face image. The 3D dynamics are then further rendered by the sparse texture mapping algorithm to recover structural details and sparse textures for generating face frames. Our model is versatile for various AR/VR and entertainment applications, such as face video retargeting and face video prediction. Superior experimental results have well demonstrated its effectiveness in generating high-fidelity, identity-preserving, and visually pleasant face video clips from a single source face image.

Video-based Facial Animation Techniques

Video-driven state-aware facial animation

Video Tracked Facial Expression Animation

Facial Expression Animation Based on Physical Model

Video-Based Facial Animation with Detailed Appearance Texture

A Survey of Computer Facial Animation Techniques

Research on Synthesis Technology of Facial Animation Driven by Kinect

A Survey of Facial Expression Generation Techniques Based on the Study Method

Interactive Facial-Geometric-Feature Animation For Generating Expressions Of Novel Faces

A data-driven approach for facial expression synthesis in video

Modeling short-term dynamics and variability for realistic interactive facial animation

Audio-driven Talking Face Video Generation with Natural Head Pose

Research on Facial Expression Simulation Based on Depth Image

Facial Feature Animation and Its Artistic Representation

Specific Face Model and Expression Animation Model Based on Physical Model

Realistic Tree-Dimensional Facial Expression Synthesis

Joint Audio-Video Driven Facial Animation

Facial Expression Transformation, Aging and Invisible View Reconstruction (1)

High Resolution Acquisition, Learning And Transfer Of Dynamic 3-D Facial Expressions

Individual facial image synthesis system for a virtual human

Image-to-Video Generation via 3D Facial Dynamics