Abstract:3D face reconstruction from multi-view video sequences has become a hotspot in computer vision for the last decades. Structure from Motion (SfM) methods, which have been widely used for multi-view 3D face reconstruction, have two main limitations. First, self-occlusion causes certain facial feature points (FFPs) to be invisible in the images, which will lead to missing data. The existing SfM methods could recover the missing data through iterative calculation, however, with high computational costs and long processing time. Second, the SfM methods cannot reconstruct the accurate 3D facial shapes of cheeks because there are no FFPs in this area. This paper proposes a novel “coarse-to-fine” multi-view 3D face reconstruction method by taking the advantage of the complementarity between FFPs and occluding contours, i.e., the boundary lines depicted between the facial region and the background. In this method, a block SfM algorithm is firstly proposed to reconstruct a “coarse” 3D facial shape by utilizing sparse FFPs. The block SfM algorithm does not estimate the true locations of the self-occluded FFPs iteratively. Thus, the computational cost is significantly reduced. Then, a kernel partial least squares (KPLS) algorithm is introduced to refine the “coarse” 3D facial shape. The KPLS method applies occluding contours to remedy the limitation of sparse FFPs correspondence-based SfM method. The proposed method is evaluated on the synthetic sequences generated from the BJUT-3D face database and the real-world multi-view video sequences obtained in a controlled indoor environment. The results show improvements in both accuracy and efficiency.

Multi-view face alignment using 3D shape model for view estimation

Face Alignment Based on 3D Face Shape Model and Markov Random Field.

Face alignment based on high order markov random field

Pose-Indexed Based Multi-View Method For Face Alignment

Automatic 3D Face Modeling Using 2D Active Appearance Models

Towards Robust and Accurate Multi-View and Partially-Occluded Face Alignment

Coarse-to-fine Multiview 3d Face Reconstruction Using Multiple Geometrical Features.

Multi-View Active Shape Model with Robust Parameter Estimation

3D3M: 3D Modulated Morphable Model for Monocular Face Reconstruction

Self-Supervised 3d Face Reconstruction Based On Multi-View Uv Fusion

Multi-view face alignment guided by several facial feature points

Multi-view Human Pose and Shape Estimation Via Mesh-Aligned Voxel Interpolation

Self-adaptive Morphable Model Based Multi-View Non-Cooperative 3D Face Reconstruction

Beyond 3DMM: Learning to Capture High-fidelity 3D Face Shape

Fast multi-view face alignment via multi-task auto-encoders

SR-VFA: Accurate Self-Refined Face Alignment in Videos.

Shape Augmented Regression For 3d Face Alignment

Face Alignment in Full Pose Range: A 3D Total Solution

Face Pose Normalization and Simulation Methods Based on Multi-View Face Alignment

An Efficient Three-Dimensional Reconstruction Approach for Pose-Invariant Face Recognition Based on a Single View.

ViewFormer: View Set Attention for Multi-view 3D Shape Understanding