Abstract:Recent face recognition algorithm can achieve high accuracy when the tested face samples are frontal. However, when the face pose changes largely, the performance of existing methods drop drastically. Efforts on pose-robust face recognition are highly desirable, especially when each face class has only one frontal training sample. In this study, we propose a 2D face fitting-assisted 3D face reconstruction algorithm that aims at recognizing faces of different poses when each face class has only one frontal training sample. For each frontal training sample, a 3D face is reconstructed by optimizing the parameters of 3D morphable model (3DMM). By rotating the reconstructed 3D face to different views, pose virtual face images are generated to enlarge the training set of face recognition. Different from the conventional 3D face reconstruction methods, the proposed algorithm utilizes automatic 2D face fitting to assist 3D face reconstruction. We automatically locate 88 sparse points of the frontal face by 2D face-fitting algorithm. Such 2D face-fitting algorithm is so-called Random Forest Embedded Active Shape Model, which embeds random forest learning into the framework of Active Shape Model. Results of 2D face fitting are added to the 3D face reconstruction objective function as shape constraints. The optimization objective energy function takes not only image intensity, but also 2D fitting results into account. Shape and texture parameters of 3DMM are thus estimated by fitting the 3DMM to the 2D frontal face sample, which is a non-linear optimization problem. We experiment the proposed method on the publicly available CMUPIE database, which includes faces viewed from 11 different poses, and the results show that the proposed method is effective and the face recognition results toward pose variants are promising.

Self-adaptive Morphable Model Based Multi-View Non-Cooperative 3D Face Reconstruction

Self-Adaptive Morphable Model Based Collaborative Multi-View 3d Face Reconstruction in Visual Sensor Network

A synthesis method for personalized 3D face reconstruction

CAMInterHand: Cooperative Attention for Multi-View Interactive Hand Pose and Mesh Reconstruction

3D3M: 3D Modulated Morphable Model for Monocular Face Reconstruction

Realistic Reconstruction of Human Face Based on Images

Coarse-to-fine Multiview 3d Face Reconstruction Using Multiple Geometrical Features.

Automatic 3D Face Modeling Using 2D Active Appearance Models

Hybrid-MVS: Robust Multi-View Reconstruction with Hybrid Optimization of Visual and Depth Cues

SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video

3d Reconstruction With Multi-View Texture Mapping

A Model Based Method for Stereo Face Reconstruction

Realistic 3D Face Modeling by Fusing Multiple 2D Images

The 3D Reconstruction of Face Model with Active Structured Light and Stereo Vision Fusion

View Consistent 3D Face Reconstruction Using Siamese Encoder-Decoders

MMFace: A Multi-Metric Regression Network for Unconstrained Face Reconstruction

2D Face Fitting-Assisted 3D Face Reconstruction for Pose-Robust Face Recognition.

Multi-view 3D Morphable Face Reconstruction via Canonical Volume Fusion

ASM: Adaptive Skinning Model for High-Quality 3D Face Modeling

Prior-Guided Multi-View 3D Head Reconstruction

Robust Geometry and Reflectance Disentanglement for 3D Face Reconstruction from Sparse-view Images