Abstract:Abstract Facial expression recognition has seen rapid development in recent years due to its wide range of applications such as human–computer interaction, health care, and social robots. Although significant progress has been made in this field, it is still challenging to recognize facial expressions with occlusions and large head-poses. To address these issues, this paper presents a cascade regression-based face frontalization (CRFF) method, which aims to immediately reconstruct a clean, frontal and expression-aware face given an in-the-wild facial image. In the first stage, a frontal facial shape is predicted by developing a cascade regression model to learn the pairwise spatial relation between non-frontal face-shape and its frontal counterpart. Unlike most existing shape prediction methods that used single-step regression, the cascade model is a multi-step regressor that gradually aligns non-frontal shape to its frontal view. We employ several different regressors and make a ensemble decision to boost prediction performance. For facial texture reconstruction, active appearance model instantiation is employed to warp the input face to the predicted frontal shape and generate a clean face. To remove occlusions, we train this generative model on manually selected clean-face sets, which ensures generating a clean face as output regardless of whether the input face involves occlusions or not. Unlike the existing face reconstruction methods that are computational expensive, the proposed method works in real time, so it is suitable for dynamic analysis of facial expression. The experimental validation shows that the ensembling cascade model has improved frontal shape prediction accuracy for an average of 5% and the proposed method has achieved superior performance on both static and dynamic recognition of facial expressions over the state-of-the-art approaches. The experimental results demonstrate that the proposed method has achieved expression-preserving frontalization, de-occlusion and has improved performance of facial expression recognition.

Face Frontalization Using an Appearance-Flow-Based Convolutional Neural Network

A Cascade Face Spoofing Detector Based on Face Anti-Spoofing R-CNN and Improved Retinex LBP

Learning Pixel Perception for Identity and Illumination Consistency Face Frontalization in the Wild.

Learning a High Fidelity Identity Representation for Face Frontalization

Facial Expression-Aware Face Frontalization.

Face Frontalization With Adaptive Soft Symmetry

Face Frontalization with Inpainting Method

Learning a High Fidelity Pose Invariant Model for High-resolution Face Frontalization

A multi-pose face frontalization method based on encoder-decoder network

A Stepwise Frontal Face Synthesis Approach for Large Pose Non-frontal Facial Image.

Cascade Support Vector Regression-Based Facial Expression-Aware Face Frontalization

3D-Guided Frontal Face Generation for Pose-Invariant Recognition

Semi-Supervised Face Frontalization in the Wild

Cascade Regression-Based Face Frontalization for Dynamic Facial Expression Analysis

Frontal Face Synthesizing According to Multiple Non-Frontal Inputs and Its Application in Face Recognition.

Pre-Trained Feature Fusion and Multidomain Identification Generative Adversarial Network for Face Frontalization.

Learning Flow-based Feature Warping for Face Frontalization with Illumination Inconsistent Supervision

Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism

Towards High Fidelity Face Frontalization in the Wild

Can We See More? Joint Frontalization and Hallucination of Unaligned Tiny Faces

3DFaceNet: Real-time Dense Face Reconstruction via Synthesizing Photo-realistic Face Images