Abstract:Facial pose variation is one of the major factors making face recognition (FR) a challenging task. One popular solution is to convert non-frontal faces to frontal ones on which FR is performed. Rotating faces causes facial pixel value changes. Therefore, existing CNN-based methods learn to synthesize frontal faces in color space. However, this learning problem in a color space is highly non-linear, causing the synthetic frontal faces to lose fine facial textures. In this paper, we take the view that the nonfrontal-frontal pixel changes are essentially caused by geometric transformations (rotation, translation, and so on) in space. Therefore, we aim to learn the nonfrontal-frontal facial conversion in the spatial domain rather than the color domain to ease the learning task. To this end, we propose an appearance-flow-based face frontalization convolutional neural network (A3F-CNN). Specifically, A3F-CNN learns to establish the dense correspondence between the non-frontal and frontal faces. Once the correspondence is built, frontal faces are synthesized by explicitly “moving” pixels from the non-frontal one. In this way, the synthetic frontal faces can preserve fine facial textures. To improve the convergence of training, an appearance-flow-guided learning strategy is proposed. In addition, generative adversarial network loss is applied to achieve a more photorealistic face, and a face mirroring method is introduced to handle the self-occlusion problem. Extensive experiments are conducted on face synthesis and pose invariant FR. Results show that our method can synthesize more photorealistic faces than the existing methods in both the controlled and uncontrolled lighting environments. Moreover, we achieve a very competitive FR performance on the Multi-PIE, LFW and IJB-A databases.

Frontal face synthesis based on improved binocular stereo vision

Frontal Face Synthesis Based on Multiple Pose-Variant Images for Face Recognition.

From Stereo Pair to 3D Face Model

An improved 3D face reconstruction method

Synthesizing Frontal Faces on Calibrated Stereo Cameras for Face Recognition

Efficient Binocular Stereo Matching Based on Sad and Improved Census Transformation

Human-Computer Interaction System Based on Binocular Stereo Vision

3D Face Reconstruction from Stereo: A Model Based Approach.

Synthesis of Face Image with Pose Variations

Stereoscopic Video Synthesis from A Monocular Video

Frontal Face Synthesizing According to Multiple Non-Frontal Inputs and Its Application in Face Recognition.

3D Face Modeling Based on Front and Side Photos

Automatic frontal view face image synthesis

A novel frontal view synthesis method based on neighbor embedding

Stereo vision based method for face location in 3D space

3D-Guided Frontal Face Generation for Pose-Invariant Recognition

Binocular Stereo Vision Technology Based on Human Visual Characteristics

Synthesizing For Face Recognition

A Model Based Method for Stereo Face Reconstruction

Face Frontalization Using an Appearance-Flow-Based Convolutional Neural Network

Stereo matching from monocular images using feature consistency