Abstract:Sensitivity to severe occlusion and large view angles limits the usage scenarios of the existing monocular 3D dense face alignment methods. The state-of-the-art 3DMM-based method, directly regresses the model's coefficients, underutilizing the low-level 2D spatial and semantic information, which can actually offer cues for face shape and orientation. In this work, we demonstrate how modeling 3D facial geometry in image and model space jointly can solve the occlusion and view angle problems. Instead of predicting the whole face directly, we regress image space features in the visible facial region by dense prediction first. Subsequently, we predict our model's coefficients based on the regressed feature of the visible regions, leveraging the prior knowledge of whole face geometry from the morphable models to complete the invisible regions. We further propose a fusion network that combines the advantages of both the image and model space predictions to achieve high robustness and accuracy in unconstrained scenarios. Thanks to the proposed fusion module, our method is robust not only to occlusion and large pitch and roll view angles, which is the benefit of our image space approach, but also to noise and large yaw angles, which is the benefit of our model space method. Comprehensive evaluations demonstrate the superior performance of our method compared with the state-of-the-art methods. On the 3D dense face alignment task, we achieve 3.80% NME on the AFLW2000-3D dataset, which outperforms the state-of-the-art method by 5.5%. Code is available at <a class="link-external link-https" href="https://github.com/lhyfst/DSFNet" rel="external noopener nofollow">this https URL</a>.

3D Dense Face Alignment with Fused Features by Aggregating CNNs and GCNs

3D Dense Face Alignment via Graph Convolution Networks

SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

End-to-End Spatial Transform Face Detection and Recognition

Multi-Scale Aggregation Network for Direct Face Alignment

Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network

Joint 3d Face Reconstruction And Dense Face Alignment Via Deep Face Feature Alignment

Towards Rich-Detail 3D Face Reconstruction and Dense Alignment Via Multi-Scale Detail Augmentation.

DSFNet: Dual Space Fusion Network for Occlusion-Robust 3D Dense Face Alignment

Face Alignment Across Large Poses: A 3D Solution

Generating 3D faces using multi-column graph convolutional networks

End-to-End 3D Face Model Reconstruction Method Based on Fusion-CNN

Learning a Multi-Center Convolutional Network for Unconstrained Face Alignment.

3D Facial Expression Recognition Using Deep Feature Fusion CNN

Towards Fast, Accurate and Stable 3D Dense Face Alignment

3D face-model reconstruction from a single image: A feature aggregation approach using hierarchical transformer with weak supervision

Face Clustering Via Adaptive Aggregation of Clean Neighbors

Attention-Guided Coarse-to-Fine Network for 2D Face Alignment in the Wild

A Coarse-to-Fine Approach for 3D Facial Landmarking by Using Deep Feature Fusion

Multi-image 3D Face Reconstruction Via an Adaptive Aggregation Network.

Pose-Invariant Face Alignment with a Single CNN