Abstract:Face reenactment aims to generate an animation of a source face using the poses and expressions from a target face. Although recent methods have made remarkable progress by exploiting generative adversarial networks, they are limited in generating high-fidelity and identity-preserving results due to the inappropriate driving information and insufficiently effective animating strategies. In this work, we propose a novel face reenactment framework that achieves both high-fidelity generation and identity preservation. Instead of sparse face representations (e.g., facial landmarks and keypoints), we utilize the Projected Normalized Coordinate Code (PNCC) to better preserve facial details. We propose to reconstruct the PNCC with the source identity parameters and the target pose and expression parameters estimated by 3D face reconstruction to factor out the target identity. By adopting the reconstructed representation as the driving information, we address the problem of identity mismatch. To effectively utilize the driving information, we establish the correspondence between the reconstructed representation and the source representation based on the features extracted by an encoder network. This identity-matched correspondence is then utilized to animate the source face using a novel feature transformation strategy. The generator network is further enhanced by the proposed geometry-aware skip connection. Once trained, our model can be applied to previously unseen faces without further training or fine-tuning. Through extensive experiments, we demonstrate the effectiveness of our method in face reenactment and show that our model outperforms state-of-the-art approaches both qualitatively and quantitatively. Additionally, the proposed PNCC reconstruction module can be easily inserted into other methods and improve their performance in cross-identity face reenactment.

Towards Accurate Facial Motion Retargeting with Identity-Consistent and Expression-Exclusive Constraints

FaceSwapNet: Landmark Guided Many-to-Many Face Reenactment

3D Facial Expression Retargeting Framework Based on an Identity-Independent Expression Feature Vector

High Resolution Acquisition, Learning And Transfer Of Dynamic 3-D Facial Expressions

Automatic facial expression recognition on a single 3D face by exploring shape deformation.

Video Tracked Facial Expression Animation

A Deformation Model to Reduce the Effect of Expressions in 3D Face Recognition

Displaced Dynamic Expression Regression for Real-Time Facial Tracking and Animation

Learning Deformation Model for Expression-Robust 3D Face Recognition

Joint Face Detection and Facial Motion Retargeting for Multiple Faces

3d Face Recognition In The Presence Of Expression: A Guidance-Based Constraint Deformation Approach

High-Fidelity Face Reenactment Via Identity-Matched Correspondence Learning

Facial Expression Re-targeting from a Single Character

Deeply Learning Deformable Facial Action Parts Model For Dynamic Expression Analysis

Jointly Optimizing Expressional and Residual Models for 3D Facial Expression Removal.

Real-Time Facial Expression Mapping for High Resolution 3D Meshes

Deep‐Learning‐Based Facial Retargeting Using Local Patches

A Data-Driven Approach for Facial Expression Retargeting in Video

Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry

High-Quality Facial Keypoints Matching With Motion Smoothness Constraint And 3d Model Constraint

3D Shape Regression for Real-Time Facial Animation