Abstract:In this paper, we present our method for neural face reenactment, called HyperReenact, that aims to generate realistic talking head images of a source identity, driven by a target facial pose. Existing state-of-the-art face reenactment methods train controllable generative models that learn to synthesize realistic facial images, yet producing reenacted faces that are prone to significant visual artifacts, especially under the challenging condition of extreme head pose changes, or requiring expensive few-shot fine-tuning to better preserve the source identity characteristics. We propose to address these limitations by leveraging the photorealistic generation ability and the disentangled properties of a pretrained StyleGAN2 generator, by first inverting the real images into its latent space and then using a hypernetwork to perform: (i) refinement of the source identity characteristics and (ii) facial pose re-targeting, eliminating this way the dependence on external editing methods that typically produce artifacts. Our method operates under the one-shot setting (i.e., using a single source frame) and allows for cross-subject reenactment, without requiring any subject-specific fine-tuning. We compare our method both quantitatively and qualitatively against several state-of-the-art techniques on the standard benchmarks of VoxCeleb1 and VoxCeleb2, demonstrating the superiority of our approach in producing artifact-free images, exhibiting remarkable robustness even under extreme head pose changes. We make the code and the pretrained models publicly available at: <a class="link-external link-https" href="https://github.com/StelaBou/HyperReenact" rel="external noopener nofollow">this https URL</a> .

Real-Time Audio-Guided Multi-Face Reenactment

APB2FaceV2: Real-Time Audio-Guided Multi-Face Reenactment

APB2FACE: Audio-Guided Face Reenactment with Auxiliary Pose and Blink Signals.

FaceSwapNet: Landmark Guided Many-to-Many Face Reenactment

FReeNet: Multi-Identity Face Reenactment

Realistic Face Reenactment Via Self-Supervised Disentangling of Identity and Pose

Audio-driven Talking Face Video Generation with Natural Head Pose

EnNeRFACE: Improving the Generalization of Face Reenactment with Adaptive Ensemble Neural Radiance Fields.

Parametric Implicit Face Representation for Audio-Driven Facial Reenactment

Face2Faceρ: Real-Time High-Resolution One-Shot Face Reenactment.

One-shot many-to-many facial reenactment using Bi-Layer Graph Convolutional Networks

Mesh Guided One-shot Face Reenactment using Graph Convolutional Networks

Face2Face<SUP></SUP>: Real-Time High-Resolution One-Shot Face Reenactment

Designing One Unified Framework for High-Fidelity Face Reenactment and Swapping

ICface: Interpretable and Controllable Face Reenactment Using GANs

FaR-GAN for One-Shot Face Reenactment

LI-Net: Large-Pose Identity-Preserving Face Reenactment Network

HyperReenact: One-Shot Reenactment via Jointly Learning to Refine and Retarget Faces

Learning Dense Correspondence for NeRF-Based Face Reenactment

RealTalk: Real-time and Realistic Audio-driven Face Generation with 3D Facial Prior-guided Identity Alignment Network

Maskrenderer: 3D-infused multi-mask realistic face reenactment