Abstract:Photorealistic multiview face synthesis from a single image is a challenging problem. Existing works mainly learn a texture mapping model from the source to the target faces. However, they rarely consider the geometric constraints on the internal deformation arising from pose variations, which causes a high level of uncertainty in face pose modeling, and hence, produces inferior results for large pose variations. Moreover, current methods typically suffer from undesired facial details loss due to the adoption of the de-facto standard encoder-decoder architecture without any skip connections (SCs). In this article, we directly learn and exploit geometric constraints and propose a fully deformable network to simultaneously model the deformations of both landmarks and faces for face synthesis. Specifically, our model consists of two parts: a deformable landmark learning network (DLLN) and a gated deformable face synthesis network (GDFSN). The DLLN converts an initial reference landmark to an individual-specific target landmark as delicate pose guidance for face rotation. The GDFSN adopts a dual-stream structure, with one stream estimating the deformation of two views in the form of convolution offsets according to the source pose and the converted target pose, and the other leveraging the predicted deformation offsets to create the target face. In this way, individual-aware pose changes are explicitly modeled in the face generator to cope with geometric transformation, by adaptively focusing on pertinent regions of the source face. To compensate for offset estimation errors, we introduce a soft-gating mechanism for adaptive fusion between deformable features and primitive features. Additionally, a pose-aligned SC (PASC) is tailored to propagate low-level input features to the appropriate positions in the output features for further enhancing the facial details and identity preservation. Extensive experiments on six benchmarks show that our approach performs favorably against the state-of-the-arts, especially with large pose changes. Code is available at https://github.com/cschengxu/FDFace.

A multi-pose face frontalization method based on encoder-decoder network

Real-Time Audio-Guided Multi-Face Reenactment

Multi-Pose Facial Expression Recognition Based on Generative Adversarial Network

Joint Face Image Restoration and Frontalization for Recognition

Face Frontalization Using an Appearance-Flow-Based Convolutional Neural Network

Frontal Face Synthesizing According to Multiple Non-Frontal Inputs and Its Application in Face Recognition.

A discriminative multiscale feature extraction network for facial expression recognition in the wild

Fully Deformable Network for Multiview Face Image Synthesis

Learning a High Fidelity Identity Representation for Face Frontalization

Multi-modal Face Anti-spoofing Using Channel Cross Fusion Network and Global Depth-Wise Convolution.

Multi-View Facial Expression Recognition with Multi-View Facial Expression Light Weight Network

Multi-Stream Facial Adaptive Network for Expression Recognition from a Single Image

Enhanced Hybrid Vision Transformer with Multi-Scale Feature Integration and Patch Dropping for Facial Expression Recognition

A Multi-View Face Expression Recognition Method Based on DenseNet and GAN

Joint Pose and Expression Modeling for Facial Expression Recognition

A multi-task approach to face deblurring

Cascade Regression-Based Face Frontalization for Dynamic Facial Expression Analysis

A Triple-Structure Network Model Based Upon MobileNet V1 and Multi-Loss Function for Facial Expression Recognition

Multi-pose Face Hallucination Via Neighbor Embedding for Facial Components

PoseFace: Pose-Invariant Features and Pose-Adaptive Loss for Face Recognition

Multi-attention Based Deep Neural Network with Hybrid Features for Dynamic Sequential Facial Expression Recognition