Abstract:Sensitivity to severe occlusion and large view angles limits the usage scenarios of the existing monocular 3D dense face alignment methods. The state-of-the-art 3DMM-based method, directly regresses the model's coefficients, underutilizing the low-level 2D spatial and semantic information, which can actually offer cues for face shape and orientation. In this work, we demonstrate how modeling 3D facial geometry in image and model space jointly can solve the occlusion and view angle problems. Instead of predicting the whole face directly, we regress image space features in the visible facial region by dense prediction first. Subsequently, we predict our model's coefficients based on the regressed feature of the visible regions, leveraging the prior knowledge of whole face geometry from the morphable models to complete the invisible regions. We further propose a fusion network that combines the advantages of both the image and model space predictions to achieve high robustness and accuracy in unconstrained scenarios. Thanks to the proposed fusion module, our method is robust not only to occlusion and large pitch and roll view angles, which is the benefit of our image space approach, but also to noise and large yaw angles, which is the benefit of our model space method. Comprehensive evaluations demonstrate the superior performance of our method compared with the state-of-the-art methods. On the 3D dense face alignment task, we achieve 3.80% NME on the AFLW2000-3D dataset, which outperforms the state-of-the-art method by 5.5%. Code is available at <a class="link-external link-https" href="https://github.com/lhyfst/DSFNet" rel="external noopener nofollow">this https URL</a>.

Deep Shape Constrained Network for Robust Face Alignment.

SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

Selective Domain-Invariant Feature Alignment Network for Face Anti-Spoofing.

Cascaded Deep Convolutional Neural Network for Robust Face Alignment.

Multistage Model for Robust Face Alignment Using Deep Neural Networks

Robust Face Alignment via Deep Progressive Reinitialization and Adaptive Error-driven Learning

Dual Sparse Constrained Cascade Regression for Robust Face Alignment.

Accurate Face Alignment Using Shape Constrained Markov Network

Effective face landmark localization via single deep network

Face Shape-Guided Deep Feature Alignment for Face Recognition Robust to Face Misalignment

Robust Face Recognition by Constrained Part-based Alignment.

Learning Deep Sharable and Structural Detectors for Face Alignment

DSFNet: Dual Space Fusion Network for Occlusion-Robust 3D Dense Face Alignment

Robust 3D Face Recognition by Local Shape Difference Boosting

Structure-Coherent Deep Feature Learning for Robust Face Alignment

ADNet: Leveraging Error-Bias Towards Normal Direction in Face Alignment

Learning Cascaded Deep Auto-Encoder Networks for Face Alignment

Face Alignment Across Large Poses: A 3D Solution

Robust Face Alignment Based on Hierarchical Classifier Network

Joint Face Alignment and 3D Face Reconstruction with Application to Face Recognition

Face Alignment with Two-Layer Shape Regression