Abstract:In this paper, we present our method for neural face reenactment, called HyperReenact, that aims to generate realistic talking head images of a source identity, driven by a target facial pose. Existing state-of-the-art face reenactment methods train controllable generative models that learn to synthesize realistic facial images, yet producing reenacted faces that are prone to significant visual artifacts, especially under the challenging condition of extreme head pose changes, or requiring expensive few-shot fine-tuning to better preserve the source identity characteristics. We propose to address these limitations by leveraging the photorealistic generation ability and the disentangled properties of a pretrained StyleGAN2 generator, by first inverting the real images into its latent space and then using a hypernetwork to perform: (i) refinement of the source identity characteristics and (ii) facial pose re-targeting, eliminating this way the dependence on external editing methods that typically produce artifacts. Our method operates under the one-shot setting (i.e., using a single source frame) and allows for cross-subject reenactment, without requiring any subject-specific fine-tuning. We compare our method both quantitatively and qualitatively against several state-of-the-art techniques on the standard benchmarks of VoxCeleb1 and VoxCeleb2, demonstrating the superiority of our approach in producing artifact-free images, exhibiting remarkable robustness even under extreme head pose changes. We make the code and the pretrained models publicly available at: <a class="link-external link-https" href="https://github.com/StelaBou/HyperReenact" rel="external noopener nofollow">this https URL</a> .

Face2Face<SUP></SUP>: Real-Time High-Resolution One-Shot Face Reenactment

Face2Faceρ: Real-Time High-Resolution One-Shot Face Reenactment.

Real-Time Audio-Guided Multi-Face Reenactment

APB2FaceV2: Real-Time Audio-Guided Multi-Face Reenactment

FaceSwapNet: Landmark Guided Many-to-Many Face Reenactment

FReeNet: Multi-Identity Face Reenactment

High-fidelity 3D Face Reconstruction with Multi-Scale Details

APB2FACE: Audio-Guided Face Reenactment with Auxiliary Pose and Blink Signals.

Designing One Unified Framework for High-Fidelity Face Reenactment and Swapping

3DFlowRenderer: One-shot Face Re-enactment via Dense 3D Facial Flow Estimation

FaR-GAN for One-Shot Face Reenactment

Mesh Guided One-shot Face Reenactment using Graph Convolutional Networks

One-shot many-to-many facial reenactment using Bi-Layer Graph Convolutional Networks

Real-time high-fidelity facial performance capture

Near-realtime Facial Animation by Deep 3D Simulation Super-Resolution

Face Replacement with Large-Pose Differences.

R2Human: Real-Time 3D Human Appearance Rendering from a Single Image

Maskrenderer: 3D-infused multi-mask realistic face reenactment

VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactment

FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping

HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Faces