Abstract:Deep learning methods have brought many breakthroughs to computer vision, especially in 2D face recognition. However, the bottleneck of deep learning based 3D face recognition is that it is difficult to collect millions of 3D faces, whether for industry or academia. In view of this situation, there are many methods to generate more 3D faces from existing 3D faces through 3D face data augmentation, which are used to train deep 3D face recognition models. However, to the best of our knowledge, there is no method to generate 3D faces from 2D face images for training deep 3D face recognition models. This letter focuses on the role of reconstructed 3D facial surfaces in 3D face identification and proposes a framework of 2D-aided deep 3D face identification. In particular, we propose to reconstruct millions of 3D face scans from a large scale 2D face database (i.e.VGGFace2), using a deep learning based 3D face reconstruction method (<a class="link-external link-http" href="http://i.e.ExpNet" rel="external noopener nofollow">this http URL</a>). Then, we adopt a two-phase training approach: In the first phase, we use millions of face images to pre-train the deep convolutional neural network (DCNN), and in the second phase, we use normal component images (NCI) of reconstructed 3D face scans to train the DCNN. Extensive experimental results illustrate that the proposed approach can greatly improve the rank-1 score of 3D face identification on the FRGC v2.0, the Bosphorus, and the BU-3DFE 3D face databases, compared to the model trained by 2D face images. Finally, our proposed approach achieves state-of-the-art rank-1 scores on the FRGC v2.0 (97.6%), Bosphorus (98.4%), and BU-3DFE (98.8%) databases. The experimental results show that the reconstructed 3D facial surfaces are useful and our 2D-aided deep 3D face identification framework is meaningful, facing the scarcity of 3D faces.

Exploring the Role of Extracted Features in Deep Learning-based 3D Face Reconstruction from Single 2D Images

3D Face Reconstruction in Deep Learning Era: A Survey

3D Face Reconstruction From Single 2D Image Using Distinctive Features

Research on 3D Face Reconstruction Algorithm Based on ResNet and Transformer

3D Face Reconstruction Based on A Single Image: A Review

Reconstructing A Large Scale 3D Face Dataset for Deep 3D Face Identification

Feature Sharing Attention 3D Face Reconstruction with Unsupervised Learning from In-the-Wild Photo Collection

Face Feature Extraction and Recognition Via Local Binary Pattern and Two-Dimensional Locality Preserving Projection

Deep Face Feature for Face Alignment and Reconstruction.

3D Face Reconstruction Based on ResNet Feature Extraction and CBAM

Survey on 3D face reconstruction from uncalibrated images

Feature Extraction and Matching of 3D Face Model Based on Facial Landmark Detection

Disentangling Features in 3D Face Shapes for Joint Face Reconstruction and Recognition

3D Face Reconstruction and Semantic Annotation from Single Depth Image

Robust Geometry and Reflectance Disentanglement for 3D Face Reconstruction from Sparse-view Images

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

3D Face Reconstruction Using A Spectral-Based Graph Convolution Encoder

CRNet:3D Face Reconstruction with Contour Map Regression Network

Beyond 3DMM Space: Towards Fine-Grained 3D Face Reconstruction

Self-supervised single-image 3D face reconstruction method based on attention mechanism and attribute refinement

FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction