Abstract:Facial pose variation is one of the major factors making face recognition (FR) a challenging task. One popular solution is to convert non-frontal faces to frontal ones on which FR is performed. Rotating faces causes facial pixel value changes. Therefore, existing CNN-based methods learn to synthesize frontal faces in color space. However, this learning problem in a color space is highly non-linear, causing the synthetic frontal faces to lose fine facial textures. In this paper, we take the view that the nonfrontal-frontal pixel changes are essentially caused by geometric transformations (rotation, translation, and so on) in space. Therefore, we aim to learn the nonfrontal-frontal facial conversion in the spatial domain rather than the color domain to ease the learning task. To this end, we propose an appearance-flow-based face frontalization convolutional neural network (A3F-CNN). Specifically, A3F-CNN learns to establish the dense correspondence between the non-frontal and frontal faces. Once the correspondence is built, frontal faces are synthesized by explicitly “moving” pixels from the non-frontal one. In this way, the synthetic frontal faces can preserve fine facial textures. To improve the convergence of training, an appearance-flow-guided learning strategy is proposed. In addition, generative adversarial network loss is applied to achieve a more photorealistic face, and a face mirroring method is introduced to handle the self-occlusion problem. Extensive experiments are conducted on face synthesis and pose invariant FR. Results show that our method can synthesize more photorealistic faces than the existing methods in both the controlled and uncontrolled lighting environments. Moreover, we achieve a very competitive FR performance on the Multi-PIE, LFW and IJB-A databases.

FF-PPQA: Face frontalization without glasses based on perceptual quality and pixel-level quality assessment

DIQA-FF:dual Image Quality Assessment for Face Frontalization

Learning Pixel Perception for Identity and Illumination Consistency Face Frontalization in the Wild.

Realistic feature perception for face frontalization with dual-mode face transformation

Automatic Eyeglasses Removal of Frontal Facial Images for Face Recognition

Joint Face Image Restoration and Frontalization for Recognition

Learning a High Fidelity Identity Representation for Face Frontalization

Face Frontalization Based on Robustly Fitting a Deformable Shape Model to 3D Landmarks

Pseudo Facial Generation With Extreme Poses for Face Recognition.

Detailed Feature Guided Generative Adversarial Pose Reconstruction Network

CP-GAN: A Cross-Pose Profile Face Frontalization Boosting Pose-Invariant Face Recognition

Face Frontalization with Inpainting Method

Expression-preserving face frontalization improves visually assisted speech processing

Robust Face Frontalization in Unconstrained Images.

Towards High Fidelity Face Frontalization in the Wild

Geometry Structure Preserving Based GAN for Multi-Pose Face Frontalization and Recognition

Frontal Face Reconstruction Based on Detail Identification, Variable Scale Self-Attention and Flexible Skip Connection

Face Hallucination With Finishing Touches

Facial Expression-Aware Face Frontalization.

Face Frontalization Using an Appearance-Flow-Based Convolutional Neural Network

PSGAN: Revisit the Binary Discriminator and an Alternative for Face Frontalization