Abstract:3D face reconstruction from multi-view video sequences has become a hotspot in computer vision for the last decades. Structure from Motion (SfM) methods, which have been widely used for multi-view 3D face reconstruction, have two main limitations. First, self-occlusion causes certain facial feature points (FFPs) to be invisible in the images, which will lead to missing data. The existing SfM methods could recover the missing data through iterative calculation, however, with high computational costs and long processing time. Second, the SfM methods cannot reconstruct the accurate 3D facial shapes of cheeks because there are no FFPs in this area. This paper proposes a novel “coarse-to-fine” multi-view 3D face reconstruction method by taking the advantage of the complementarity between FFPs and occluding contours, i.e., the boundary lines depicted between the facial region and the background. In this method, a block SfM algorithm is firstly proposed to reconstruct a “coarse” 3D facial shape by utilizing sparse FFPs. The block SfM algorithm does not estimate the true locations of the self-occluded FFPs iteratively. Thus, the computational cost is significantly reduced. Then, a kernel partial least squares (KPLS) algorithm is introduced to refine the “coarse” 3D facial shape. The KPLS method applies occluding contours to remedy the limitation of sparse FFPs correspondence-based SfM method. The proposed method is evaluated on the synthetic sequences generated from the BJUT-3D face database and the real-world multi-view video sequences obtained in a controlled indoor environment. The results show improvements in both accuracy and efficiency.

Model-Based Multi-view Face Construction and Recognition in Videos

Multi-view face recognition based on tensor subspace analysis and view manifold modeling

Joint Structured Sparsity Regularized Multiview Dimension Reduction for Video-Based Facial Expression Recognition.

Multi-Pose Face Recognition Based on Orthogonal Views

Multi-person Multi-Camera Tracking for Live Stream Videos Based on Improved Motion Model and Matching Cascade

Multiview Face Recognition: from TensorFace to V-TensorFace and K-TensorFace

Coarse-to-fine Multiview 3d Face Reconstruction Using Multiple Geometrical Features.

A Novel Multi-View Dimensionality Reduction and Recognition Framework with Applications to Face Recognition.

Human Multi-View Synthesis from a Single-View Model:Transferred Body and Face Representations

Nonlinear View Manifold Modeling Methods for Multi-View Faces

Self-adaptive Morphable Model Based Multi-View Non-Cooperative 3D Face Reconstruction

Deep Learning Multi-View Representation for Face Recognition

Constrained Multi-View Video Face Clustering.

Locality-constrained Linear Coding Based Bi-Layer Model for Multi-View Facial Expression Recognition

A New Method for Multi-view Face Clustering in Video Sequence

Temporal-spatial Face Recognition Using Multi-Atlas and Markov Process Model

Multi-view face alignment using 3D shape model for view estimation

Multi-view Face Recognition by Nonlinear Tensor Decomposition

A Now Multi-view Face Tracking Algorithm

Pose-invariant face recognition based on a single view

Multi-view face recognition based on manifold learning and multilinear representation