Abstract:Recently proposed robust 3D face alignment methods establish either dense or sparse correspondence between a 3D face model and a 2D facial image. The use of these methods presents new challenges as well as opportunities for facial texture analysis. In particular, by sampling the image using the fitted model, a facial UV can be created. Unfortunately, due to self-occlusion, such a UV map is always incomplete. In this paper, we propose a framework for training Deep Convolutional Neural Network (DCNN) to complete the facial UV map extracted from in-the-wild images. To this end, we first gather complete UV maps by fitting a 3D Morphable Model (3DMM) to various multiview image and video datasets, as well as leveraging on a new 3D dataset with over 3,000 identities. Second, we devise a meticulously designed architecture that combines local and global adversarial DCNNs to learn an identity-preserving facial UV completion model. We demonstrate that by attaching the completed UV to the fitted mesh and generating instances of arbitrary poses, we can increase pose variations for training deep face recognition/verification models, and minimise pose discrepancy during testing, which lead to better performance. Experiments on both controlled and in-the-wild UV datasets prove the effectiveness of our adversarial UV completion model. We achieve state-of-the-art verification accuracy, $94.05\%$, under the CFP frontal-profile protocol only by combining pose augmentation during training and pose discrepancy reduction during testing. We will release the first in-the-wild UV dataset (we refer as WildUV) that comprises of complete facial UV maps from 1,892 identities for research purposes.

UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition

Towards high-fidelity facial UV map generation in real-world

SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

Unified Video and Image Representation for Boosted Video Face Forgery Detection

Multi-Attribute Robust Component Analysis for Facial UV Maps

High-Quality 3D Face Reconstruction with Affine Convolutional Networks

3D-Aware Adversarial Makeup Generation for Facial Privacy Protection

Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network

Deep Face Video Inpainting via UV Mapping

Pose2UV: Single-Shot Multiperson Mesh Recovery with Deep UV Prior.

Self-Supervised 3d Face Reconstruction Based On Multi-View Uv Fusion

Learning Dense UV Completion for Human Mesh Recovery

3DFP-FCGAN: Face Completion Generative Adversarial Network with 3D Facial Prior

IFaceUV: Intuitive Motion Facial Image Generation by Identity Preservation via UV map

Beyond 3DMM Space: Towards Fine-Grained 3D Face Reconstruction

Semi-supervised Adversarial Learning to Generate Photorealistic Face Images of New Identities from 3D Morphable Model

FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstruction.

Face Recognition Using a Unified 3D Morphable Model

Fine-grained Face Verification: FGLFW Database, Baselines, and Human-Dcmn Partnership.

Full Face Texture Generation of Virtual Human