Abstract:Deep learning methods have brought many breakthroughs to computer vision, especially in 2D face recognition. However, the bottleneck of deep learning based 3D face recognition is that it is difficult to collect millions of 3D faces, whether for industry or academia. In view of this situation, there are many methods to generate more 3D faces from existing 3D faces through 3D face data augmentation, which are used to train deep 3D face recognition models. However, to the best of our knowledge, there is no method to generate 3D faces from 2D face images for training deep 3D face recognition models. This letter focuses on the role of reconstructed 3D facial surfaces in 3D face identification and proposes a framework of 2D-aided deep 3D face identification. In particular, we propose to reconstruct millions of 3D face scans from a large scale 2D face database (i.e.VGGFace2), using a deep learning based 3D face reconstruction method (<a class="link-external link-http" href="http://i.e.ExpNet" rel="external noopener nofollow">this http URL</a>). Then, we adopt a two-phase training approach: In the first phase, we use millions of face images to pre-train the deep convolutional neural network (DCNN), and in the second phase, we use normal component images (NCI) of reconstructed 3D face scans to train the DCNN. Extensive experimental results illustrate that the proposed approach can greatly improve the rank-1 score of 3D face identification on the FRGC v2.0, the Bosphorus, and the BU-3DFE 3D face databases, compared to the model trained by 2D face images. Finally, our proposed approach achieves state-of-the-art rank-1 scores on the FRGC v2.0 (97.6%), Bosphorus (98.4%), and BU-3DFE (98.8%) databases. The experimental results show that the reconstructed 3D facial surfaces are useful and our 2D-aided deep 3D face identification framework is meaningful, facing the scarcity of 3D faces.

Recognition-oriented facial depth estimation from a single image

Depth Generation Network: Estimating Real World Depth From Stereo And Depth Images

Improving RGB-D Face Recognition via Transfer Learning from a Pretrained 2D Network.

Facial depth descend: A generation paradigm for facial depth map

Face Denoising and 3D Reconstruction from A Single Depth Image

Depth-Based 3D Face Reconstruction and Pose Estimation Using Shape-Preserving Domain Adaptation

Reconstructing A Large Scale 3D Face Dataset for Deep 3D Face Identification

High-Accuracy RGB-D Face Recognition via Segmentation-Aware Face Depth Estimation and Mask-Guided Attention Network

Depth Images Could Tell Us More: Enhancing Depth Discriminability for RGB-D Scene Recognition

Robust 3D Face Recognition by Local Shape Difference Boosting

Few-Data Guided Learning Upon End-to-End Point Cloud Network for 3D Face Recognition

Two Birds with One Stone: Iteratively Learn Facial Attributes with GANs.

Facial Depth and Normal Estimation using Single Dual-Pixel Camera

3D Face Recognition using Mapped Depth Images

Multi-channel Deep 3D Face Recognition

Facial Expression Recognition with an Attention Network Using a Single Depth Image

Deep Learning-based Single Image Face Depth Data Enhancement

Data-Free Point Cloud Network for 3D Face Recognition

3D Face Reconstruction Based on ResNet Feature Extraction and CBAM

NormalGAN: Learning Detailed 3D Human from a Single RGB-D Image

FA-GAN: Face Augmentation GAN for Deformation-Invariant Face Recognition