Abstract:3D face reconstruction from multi-view video sequences has become a hotspot in computer vision for the last decades. Structure from Motion (SfM) methods, which have been widely used for multi-view 3D face reconstruction, have two main limitations. First, self-occlusion causes certain facial feature points (FFPs) to be invisible in the images, which will lead to missing data. The existing SfM methods could recover the missing data through iterative calculation, however, with high computational costs and long processing time. Second, the SfM methods cannot reconstruct the accurate 3D facial shapes of cheeks because there are no FFPs in this area. This paper proposes a novel “coarse-to-fine” multi-view 3D face reconstruction method by taking the advantage of the complementarity between FFPs and occluding contours, i.e., the boundary lines depicted between the facial region and the background. In this method, a block SfM algorithm is firstly proposed to reconstruct a “coarse” 3D facial shape by utilizing sparse FFPs. The block SfM algorithm does not estimate the true locations of the self-occluded FFPs iteratively. Thus, the computational cost is significantly reduced. Then, a kernel partial least squares (KPLS) algorithm is introduced to refine the “coarse” 3D facial shape. The KPLS method applies occluding contours to remedy the limitation of sparse FFPs correspondence-based SfM method. The proposed method is evaluated on the synthetic sequences generated from the BJUT-3D face database and the real-world multi-view video sequences obtained in a controlled indoor environment. The results show improvements in both accuracy and efficiency.

A Sfm-Based Sparse To Dense 3d Face Reconstruction Method Robust To Feature Tracking Errors

A synthesis method for personalized 3D face reconstruction

Local Spatial Continuity Steered Sparse Representation for Occluded Face Recognition

A Simple 3d Face Tracking Method Based On Depth Information

Coarse-to-fine Multiview 3d Face Reconstruction Using Multiple Geometrical Features.

Robust 3D Reconstruction with an RGB-D Camera

TC-SfM: Robust Track-Community-Based Structure-from-Motion

The 3D Reconstruction of Face Model with Active Structured Light and Stereo Vision Fusion

3D Facial Geometry Recovery Via Group-Wise Optical Flow

Robust Real-Time Face Tracking And Modeling From Video

CRF-Based Reconstruction from Narrow-Baseline Image Sequences.

Feature-Assisted Dense Spatio-Temporal Reconstruction From Binocular Sequences

3D Face Tracking from 2D Video through Iterative Dense UV to Image Flow

3D Face Sparse Reconstruction Based on Local Linear Fitting.

RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairs

Robust 3D Reconstruction from Uncalibrated Small Motion Clips

Fast and Robust Feature Tracking for 3D Reconstruction

HeadRecon: High-Fidelity 3D Head Reconstruction from Monocular Video

3D3M: 3D Modulated Morphable Model for Monocular Face Reconstruction

A Real-Time Method for Recovering 3D Faces from Monocular Video

Incremental SFM 3D Reconstruction Based on Deep Learning