Abstract:Pose variation and occlusion are two key factors that affect the accuracy of face recognition. Most of the previous work alleviate the impacts of pose and occlusion by performing the tasks of face frontalization and face completion, respectively. Specially, generative adversarial networks (GANs) based methods have made a great progress on both of these two tasks. However, the two tasks are rarely paid attention simultaneously. Hence, the synthesis and recognition from the profile but occluded facial image is still an understudied and challenging problem in two aspects. 1) Occlusion mask, as a kind of noise, can be some very important prior information in the corrupted image. In particular, the occlusion mask is often used to fit this noise and help face restoration of the occluded region. However, a prior work, such as BoostGAN, failed to utilize the mask guided noise prior information. 2) The two tasks, de-occlusion and frontalization, are collaborative, so the identity discriminative information is easily lost if the two tasks are not organically unified in the training phase. In order to overcome these challenges, we propose a novel mask guided two-stage generative adversarial network (TSGAN). There are two major contributions in this work: 1) In order to utilize the prior information of noise, an occluded mask based attention model is proposed, which is integrated into the two stages via U-connection. This mask works as a guidance to simultaneously repair and frontalize the profile but occluded faces. 2) An end-to-end paradigm with a two-stage architecture (i.e., face deocclusion network and face frontalization network) is proposed to complete the two tasks jointly. Furthermore, for preserving the discriminative identity information in both stages more effectively, we propose a novel dual triplet loss, consisting of a deocclusion triplet loss and a frontalization triplet loss. Qualitative and quantitative experiments on both constr-ined and unconstrained face datasets with regular and irregular (natural) occlusions demonstrate the superiority of our approach.

SD-GAN: Structural and Denoising GAN reveals facial parts under occlusion

Look More into Occlusion: Realistic Face Frontalization and Recognition with BoostGAN.

Semi-Supervised Natural Face De-Occlusion.

Self-Supervised GAN for Occluded Facial Expression Recognition

Simultaneous Face Completion and Frontalization via Mask Guided Two-Stage GAN

SD-GAN: Semantic Decomposition for Face Image Synthesis with Discrete Attribute

Two Birds with One Stone: Iteratively Learn Facial Attributes with GANs.

FD-StackGAN: Face De-occlusion Using Stacked Generative Adversarial Networks.

BoostGAN for Occlusive Profile Face Frontalization and Recognition

Deep Generative Adversarial Network for Occlusion Removal from a Single Image

Two Birds with One Stone: Transforming and Generating Facial Images with Iterative GAN

dc-GAN: Dual-Conditioned GAN for Face Demorphing From a Single Morph

Reconstruct Face from Features Using GAN Generator as a Distribution Constraint

Geometry Structure Preserving Based GAN for Multi-Pose Face Frontalization and Recognition

DSU-GAN: A Robust Frontal Face Recognition Approach Based on Generative Adversarial Network

Semantic Text-to-Face GAN -ST^2FG

Improving Face Recognition by Integrating Decision Forest into GAN

On Hallucinating Context and Background Pixels from a Face Mask using Multi-scale GANs

Semantic-aware Deidentification Generative Adversarial Networks for Identity Anonymization

Face Merged Generative Adversarial Network with Tripartite Adversaries

FA-GAN: Face Augmentation GAN for Deformation-Invariant Face Recognition