Abstract:Photorealistic multiview face synthesis from a single image is a challenging problem. Existing works mainly learn a texture mapping model from the source to the target faces. However, they rarely consider the geometric constraints on the internal deformation arising from pose variations, which causes a high level of uncertainty in face pose modeling, and hence, produces inferior results for large pose variations. Moreover, current methods typically suffer from undesired facial details loss due to the adoption of the de-facto standard encoder-decoder architecture without any skip connections (SCs). In this article, we directly learn and exploit geometric constraints and propose a fully deformable network to simultaneously model the deformations of both landmarks and faces for face synthesis. Specifically, our model consists of two parts: a deformable landmark learning network (DLLN) and a gated deformable face synthesis network (GDFSN). The DLLN converts an initial reference landmark to an individual-specific target landmark as delicate pose guidance for face rotation. The GDFSN adopts a dual-stream structure, with one stream estimating the deformation of two views in the form of convolution offsets according to the source pose and the converted target pose, and the other leveraging the predicted deformation offsets to create the target face. In this way, individual-aware pose changes are explicitly modeled in the face generator to cope with geometric transformation, by adaptively focusing on pertinent regions of the source face. To compensate for offset estimation errors, we introduce a soft-gating mechanism for adaptive fusion between deformable features and primitive features. Additionally, a pose-aligned SC (PASC) is tailored to propagate low-level input features to the appropriate positions in the output features for further enhancing the facial details and identity preservation. Extensive experiments on six benchmarks show that our approach performs favorably against the state-of-the-arts, especially with large pose changes. Code is available at https://github.com/cschengxu/FDFace.

A Deep Variational Autoencoder Approach for Robust Facial Symmetrization

Symmetrical Singular Value Decomposition Representation for Pattern Recognition

Region-Aware Face Swapping

Facial Landmark Disentangled Network with Variational Autoencoder

Realistic Face Reenactment Via Self-Supervised Disentangling of Identity and Pose

Semantically Disentangled Variational Autoencoder for Modeling 3D Facial Details

SymFace: Additional Facial Symmetry Loss for Deep Face Recognition

Automatic Quantification of Facial Asymmetry using Facial Landmarks

Latent Disentanglement in Mesh Variational Autoencoders Improves the Diagnosis of Craniofacial Syndromes and Aids Surgical Planning

Generative Adversarial Network-based Synthesis of Visible Faces from Polarimetric Thermal Faces

Face Synthesis from Visual Attributes via Sketch using Conditional VAEs and GANs

Identity-aware Facial Expression Recognition via Deep Metric Learning based on Synthesized Images

Fully Deformable Network for Multiview Face Image Synthesis

Facial expression recognition and generation using sparse autoencoder

Robust Symmetry Detection via Riemannian Langevin Dynamics

3D Face Modeling via Weakly-supervised Disentanglement Network joint Identity-consistency Prior

3D GAN Inversion with Facial Symmetry Prior

Synthesis of High-Quality Visible Faces from Polarimetric Thermal Faces using Generative Adversarial Networks

Facial Synthesis from Visual Attributes via Sketch using Multi-Scale Generators

Using the original and 'symmetrical face' training samples to perform representation based two-step face recognition

Joint Expression Synthesis and Representation Learning for Facial Expression Recognition