Abstract:Face reenactment aims to generate an animation of a source face using the poses and expressions from a target face. Although recent methods have made remarkable progress by exploiting generative adversarial networks, they are limited in generating high-fidelity and identity-preserving results due to the inappropriate driving information and insufficiently effective animating strategies. In this work, we propose a novel face reenactment framework that achieves both high-fidelity generation and identity preservation. Instead of sparse face representations (e.g., facial landmarks and keypoints), we utilize the Projected Normalized Coordinate Code (PNCC) to better preserve facial details. We propose to reconstruct the PNCC with the source identity parameters and the target pose and expression parameters estimated by 3D face reconstruction to factor out the target identity. By adopting the reconstructed representation as the driving information, we address the problem of identity mismatch. To effectively utilize the driving information, we establish the correspondence between the reconstructed representation and the source representation based on the features extracted by an encoder network. This identity-matched correspondence is then utilized to animate the source face using a novel feature transformation strategy. The generator network is further enhanced by the proposed geometry-aware skip connection. Once trained, our model can be applied to previously unseen faces without further training or fine-tuning. Through extensive experiments, we demonstrate the effectiveness of our method in face reenactment and show that our model outperforms state-of-the-art approaches both qualitatively and quantitatively. Additionally, the proposed PNCC reconstruction module can be easily inserted into other methods and improve their performance in cross-identity face reenactment.

Face2Faceρ: Real-Time High-Resolution One-Shot Face Reenactment.

Face2Face<SUP></SUP>: Real-Time High-Resolution One-Shot Face Reenactment

Real-Time Audio-Guided Multi-Face Reenactment

APB2FaceV2: Real-Time Audio-Guided Multi-Face Reenactment

FaceSwapNet: Landmark Guided Many-to-Many Face Reenactment

FReeNet: Multi-Identity Face Reenactment

APB2FACE: Audio-Guided Face Reenactment with Auxiliary Pose and Blink Signals.

High-Fidelity Face Reenactment Via Identity-Matched Correspondence Learning

Large Pose Friendly Face Reenactment using subtle motions

FaR-GAN for One-Shot Face Reenactment

High Fidelity Motion Adaptive Face Reenactment.

Mesh Guided One-shot Face Reenactment using Graph Convolutional Networks

3DFlowRenderer: One-shot Face Re-enactment via Dense 3D Facial Flow Estimation

Designing One Unified Framework for High-Fidelity Face Reenactment and Swapping

CNN-Based Real-Time Dense Face Reconstruction with Inverse-Rendered Photo-Realistic Face Images.

One-shot many-to-many facial reenactment using Bi-Layer Graph Convolutional Networks

Maskrenderer: 3D-infused multi-mask realistic face reenactment

HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Faces

3DFaceNet: Real-time Dense Face Reconstruction via Synthesizing Photo-realistic Face Images

A Two-Stage Framework for 3D FaceReconstruction from RGBD Images

Semantic-aware One-shot Face Re-enactment with Dense Correspondence Estimation