Abstract:Photorealistic multiview face synthesis from a single image is a challenging problem. Existing works mainly learn a texture mapping model from the source to the target faces. However, they rarely consider the geometric constraints on the internal deformation arising from pose variations, which causes a high level of uncertainty in face pose modeling, and hence, produces inferior results for large pose variations. Moreover, current methods typically suffer from undesired facial details loss due to the adoption of the de-facto standard encoder-decoder architecture without any skip connections (SCs). In this article, we directly learn and exploit geometric constraints and propose a fully deformable network to simultaneously model the deformations of both landmarks and faces for face synthesis. Specifically, our model consists of two parts: a deformable landmark learning network (DLLN) and a gated deformable face synthesis network (GDFSN). The DLLN converts an initial reference landmark to an individual-specific target landmark as delicate pose guidance for face rotation. The GDFSN adopts a dual-stream structure, with one stream estimating the deformation of two views in the form of convolution offsets according to the source pose and the converted target pose, and the other leveraging the predicted deformation offsets to create the target face. In this way, individual-aware pose changes are explicitly modeled in the face generator to cope with geometric transformation, by adaptively focusing on pertinent regions of the source face. To compensate for offset estimation errors, we introduce a soft-gating mechanism for adaptive fusion between deformable features and primitive features. Additionally, a pose-aligned SC (PASC) is tailored to propagate low-level input features to the appropriate positions in the output features for further enhancing the facial details and identity preservation. Extensive experiments on six benchmarks show that our approach performs favorably against the state-of-the-arts, especially with large pose changes. Code is available at https://github.com/cschengxu/FDFace.

Deep Disentangling Siamese Network for Frontal Face Synthesis under Neutral Illumination

A Symmetrical Siamese Network Framework with Contrastive Learning for Pose-Robust Face Recognition.

A multi-pose face frontalization method based on encoder-decoder network

Fully Deformable Network for Multiview Face Image Synthesis

Deep transfer network for face recognition using 3D synthesized face

Occlusion Robust Face Recognition Based on Mask Learning with Pairwise Differential Siamese Network

DP-Face: Privacy-Preserving Face Recognition Using Siamese Network

The Devil is in the Detail: Deep Feature Based Disguised Face Recognition Method

Deep Transfer Network with 3D Morphable Models for Face Recognition

SynFace: Face Recognition with Synthetic Data

Deep Graphical Feature Learning for Face Sketch Synthesis.

LiSiam: Localization Invariance Siamese Network for Deepfake Detection

A Siamese Network for Face Verification with Depth Images

Reinforced Disentanglement for Face Swapping Without Skip Connection

Efficient face manipulation via deep feature disentanglement and reintegration net

Toward Identity Preserving Face Synthesis Between Sketches and Photos Using Deep Feature Injection

DSNet: Dual-stream Multi-Scale Fusion Network for Low-Quality 3D Face Recognition

Semi-Siamese Training for Shallow Face Learning

DSU-GAN: A Robust Frontal Face Recognition Approach Based on Generative Adversarial Network

A Novel Frontal Facial Synthesis Algorithm Based on Individual Residual Face.

DR-Net: A Multi-View Face Synthesis Network Driven by Dual Representation