Abstract:Abstract Facial expression recognition has seen rapid development in recent years due to its wide range of applications such as human–computer interaction, health care, and social robots. Although significant progress has been made in this field, it is still challenging to recognize facial expressions with occlusions and large head-poses. To address these issues, this paper presents a cascade regression-based face frontalization (CRFF) method, which aims to immediately reconstruct a clean, frontal and expression-aware face given an in-the-wild facial image. In the first stage, a frontal facial shape is predicted by developing a cascade regression model to learn the pairwise spatial relation between non-frontal face-shape and its frontal counterpart. Unlike most existing shape prediction methods that used single-step regression, the cascade model is a multi-step regressor that gradually aligns non-frontal shape to its frontal view. We employ several different regressors and make a ensemble decision to boost prediction performance. For facial texture reconstruction, active appearance model instantiation is employed to warp the input face to the predicted frontal shape and generate a clean face. To remove occlusions, we train this generative model on manually selected clean-face sets, which ensures generating a clean face as output regardless of whether the input face involves occlusions or not. Unlike the existing face reconstruction methods that are computational expensive, the proposed method works in real time, so it is suitable for dynamic analysis of facial expression. The experimental validation shows that the ensembling cascade model has improved frontal shape prediction accuracy for an average of 5% and the proposed method has achieved superior performance on both static and dynamic recognition of facial expressions over the state-of-the-art approaches. The experimental results demonstrate that the proposed method has achieved expression-preserving frontalization, de-occlusion and has improved performance of facial expression recognition.

Dual Sparse Constrained Cascade Regression for Robust Face Alignment.

Adaptive Cascade Regression Model for Robust Face Alignment.

SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

Cascade of Forests for Face Alignment

A Cross-Dimension Annotations Method for 3D Structural Facial Landmark Extraction

Local Spatial Continuity Steered Sparse Representation for Occluded Face Recognition

Constrained Joint Cascade Regression Framework for Simultaneous Facial Action Unit Recognition and Facial Landmark Detection

Robust Face Recognition by Constrained Part-based Alignment.

Face Alignment Based on Two-Stage Localization

Face Alignment with Two-Layer Shape Regression

M3 CSR: Multi-view, multi-scale and multi-component cascade shape regression

Cascaded Deep Convolutional Neural Network for Robust Face Alignment.

Dynamic Cascaded Regression Network with Reinforcement Learning for Robust Face Alignment

Robust Face Alignment via Deep Progressive Reinitialization and Adaptive Error-driven Learning

Efficient and Accurate Face Alignment by Global Regression and Cascaded Local Refinement.

Multistage Model for Robust Face Alignment Using Deep Neural Networks

Robust 3D Face Recognition by Local Shape Difference Boosting

Robust Face Recognition Via Accurate Face Alignment and Sparse Representation

Learning Deep Sharable and Structural Detectors for Face Alignment

Cascade Regression-Based Face Frontalization for Dynamic Facial Expression Analysis

Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognition