Abstract:Recent researches reveal that StyleGAN can generate highly realistic images, inspiring researchers to use pretrained StyleGAN to generate high-fidelity swapped faces. However, existing methods fail to meet the expectations in two essential aspects of high-fidelity face swapping. Their results are blurry without pore-level details and fail to preserve identity for challenging cases. To overcome the above artifacts, we innovatively construct a series of identity-preserving semantic bases of StyleGAN (called StyleIPSB) in respect of pose, expression, and illumination. Each basis of StyleIPSB controls one specific semantic attribute and disentangles with the others. The StyleIPSB constrains style code in the subspace of W+ space to preserve pore-level details and gives us a novel tool for high-fidelity face swapping, and we propose a three-stage framework for face swapping with StyleIPSB. Firstly, we transform the target facial images' attributes to the source image. We learn the mapping from 3D Morphable Model (3DMM) parameters, which capture the prominent semantic variance, to the coordinates of StyleIPSB that show higher identity-preserving and fidelity. Secondly, to transform detailed attributes which 3DMM does not capture, we learn the residual attribute between the reenacted face and the target face. Finally, the face is blended into the background of the target image. Extensive results and comparisons demonstrate that StyleIPSB can effectively preserve identity and pore-level details. The results of face swapping can achieve state-of-the-art performance. We will release our code at https://github.com/a686432/StyleIPSB

Supplementary Material for 3D-Aware Face Swapping

FaceSwapNet: Landmark Guided Many-to-Many Face Reenactment

3D-Aware Face Swapping

Region-Aware Face Swapping

Realistic Face Reenactment Via Self-Supervised Disentangling of Identity and Pose

Fine-Grained Face Swapping via Regional GAN Inversion

High-Fidelity Face Swapping with Style Blending

FSGANv2: Improved Subject Agnostic Face Swapping and Reenactment

Designing One Unified Framework for High-Fidelity Face Reenactment and Swapping

E4S: Fine-grained Face Swapping via Editing With Regional GAN Inversion

On Face Segmentation, Face Swapping, and Face Perception

Reinforced Disentanglement for Face Swapping Without Skip Connection

Face Swapping under Large Pose Variations: A 3D Model Based Approach

StyleIPSB: Identity-Preserving Semantic Basis of StyleGAN for High Fidelity Face Swapping

HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping

High-resolution Face Swapping via Latent Semantics Disentanglement

Face swapping based on 3-D photo realistic head reconstruction

FSGAN: Subject Agnostic Face Swapping and Reenactment

Realistic and Efficient Face Swapping: A Unified Approach with Diffusion Models

Deep Face Swapping via Cross-Identity Adversarial Training.

Identity-Preserving Face Swapping via Dual Surrogate Generative Models