Abstract:Photorealistic multiview face synthesis from a single image is a challenging problem. Existing works mainly learn a texture mapping model from the source to the target faces. However, they rarely consider the geometric constraints on the internal deformation arising from pose variations, which causes a high level of uncertainty in face pose modeling, and hence, produces inferior results for large pose variations. Moreover, current methods typically suffer from undesired facial details loss due to the adoption of the de-facto standard encoder-decoder architecture without any skip connections (SCs). In this article, we directly learn and exploit geometric constraints and propose a fully deformable network to simultaneously model the deformations of both landmarks and faces for face synthesis. Specifically, our model consists of two parts: a deformable landmark learning network (DLLN) and a gated deformable face synthesis network (GDFSN). The DLLN converts an initial reference landmark to an individual-specific target landmark as delicate pose guidance for face rotation. The GDFSN adopts a dual-stream structure, with one stream estimating the deformation of two views in the form of convolution offsets according to the source pose and the converted target pose, and the other leveraging the predicted deformation offsets to create the target face. In this way, individual-aware pose changes are explicitly modeled in the face generator to cope with geometric transformation, by adaptively focusing on pertinent regions of the source face. To compensate for offset estimation errors, we introduce a soft-gating mechanism for adaptive fusion between deformable features and primitive features. Additionally, a pose-aligned SC (PASC) is tailored to propagate low-level input features to the appropriate positions in the output features for further enhancing the facial details and identity preservation. Extensive experiments on six benchmarks show that our approach performs favorably against the state-of-the-arts, especially with large pose changes. Code is available at https://github.com/cschengxu/FDFace.

A face template: Improving the face generation quality of multi-stage generative adversarial networks using coarse-grained facial priors

Two Birds with One Stone: Transforming and Generating Facial Images with Iterative GAN

Text-to-face Synthesis Based on Facial Landmarks Prediction

Two Birds with One Stone: Iteratively Learn Facial Attributes with GANs.

FTGAN: A Fully-trained Generative Adversarial Networks for Text to Face Generation

Multi-Modal Face Stylization with a Generative Prior

An improved StyleGAN-based TextToFace model with Local-Global information Fusion

MHGAN: Multi-Hierarchies Generative Adversarial Network for High-Quality Face Sketch Synthesis

Progressive Text-to-Face Synthesis with Generative Adversarial Network

Fine-Granularity Face Sketch Synthesis

Structured Detail Enhancement for Cross-Modality Face Synthesis

Deep Face Generation from a Rough Sketch Using Multi-Level Generative Adversarial Networks

Face Sketch Synthesis via Semantic-Driven Generative Adversarial Network

DualG-GAN, a Dual-channel Generator based Generative Adversarial Network for text-to-face synthesis

HCGAN: hierarchical contrast generative adversarial network for unpaired sketch face synthesis

High-Fidelity Face Sketch-To-Photo Synthesis Using Generative Adversarial Network

Fully Deformable Network for Multiview Face Image Synthesis

Controllable Face Sketch-Photo Synthesis with Flexible Generative Priors

FL-GAN: Feature Learning Generative Adversarial Network for High-Quality Face Sketch Synthesis

Face Merged Generative Adversarial Network with Tripartite Adversaries

AnyFace++: A Unified Framework for Free-style Text-to-Face Synthesis and Manipulation.