Abstract:In this paper, we present a large-scale detailed 3D face dataset, FaceScape, and the corresponding benchmark to evaluate single-view facial 3D reconstruction. By training on FaceScape data, a novel algorithm is proposed to predict elaborate riggable 3D face models from a single image input. FaceScape dataset releases $16,940$ textured 3D faces, captured from $847$ subjects and each with $20$ specific expressions. The 3D models contain the pore-level facial geometry that is also processed to be topologically uniform. These fine 3D facial models can be represented as a 3D morphable model for coarse shapes and displacement maps for detailed geometry. Taking advantage of the large-scale and high-accuracy dataset, a novel algorithm is further proposed to learn the expression-specific dynamic details using a deep neural network. The learned relationship serves as the foundation of our 3D face prediction system from a single image input. Different from most previous methods, our predicted 3D models are riggable with highly detailed geometry under different expressions. We also use FaceScape data to generate the in-the-wild and in-the-lab benchmark to evaluate recent methods of single-view face reconstruction. The accuracy is reported and analyzed on the dimensions of camera pose and focal length, which provides a faithful and comprehensive evaluation and reveals new challenges. The unprecedented dataset, benchmark, and code have been released at <a class="link-external link-https" href="https://github.com/zhuhao-nju/facescape" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The paper attempts to address several key challenges in Single-View 3D Face Reconstruction. Specifically: 1. **Lack of large-scale high-quality 3D face datasets**: Existing 3D face datasets are limited in scale and quality, which restricts the development of facial analysis and reconstruction research. 2. **Predicting detailed and deformable 3D face models**: Most existing methods cannot predict highly detailed and deformable 3D face models from a single image. 3. **Standards for evaluating single-view 3D face reconstruction methods**: There is a lack of a comprehensive benchmark to evaluate the performance of different methods in single-view 3D face reconstruction tasks. To address these issues, the paper proposes the following contributions: 1. **Constructing a large-scale detailed 3D face dataset FaceScape**: It contains 16,940 textured 3D face models from 847 subjects, with 20 specific expressions for each subject. These 3D models have pore-level facial geometry and have been topologically uniformed. 2. **Proposing a two-stage pipeline to predict detailed and deformable 3D face models**: First, a coarse mesh model is fitted based on detected 2D landmarks, then a displacement map for each expression is predicted, forming a mid-level geometric representation in a linear space. Unlike previous static geometric detail prediction methods, this method can deform the predicted details into any expression. 3. **Establishing a new benchmark**: It includes real-world and laboratory data for evaluating single-view 3D face reconstruction methods. This benchmark comprehensively evaluates 14 state-of-the-art methods in dimensions such as camera pose and focal length, revealing new challenges. Through these contributions, the paper aims to advance the technology of single-view 3D face reconstruction and provide high-quality data and evaluation standards for related research.

FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction

FaceScape: a Large-scale High Quality 3D Face Dataset and Detailed Riggable 3D Face Prediction

Pixel-Face: A Large-Scale, High-Resolution Benchmark for 3D Face Reconstruction

RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairs

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

FaceLift: Single Image to 3D Head with View Generation and GS-LRM

Joint 3d Face Reconstruction And Dense Face Alignment Via Deep Face Feature Alignment

3D Face Reconstruction and Semantic Annotation from Single Depth Image

SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

Beyond 3DMM Space: Towards Fine-Grained 3D Face Reconstruction

3DFaceNet: Real-time Dense Face Reconstruction via Synthesizing Photo-realistic Face Images

Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set

3D Face Reconstruction System Based on Deep Learning and Sparse Face Model

3D Face Reconstruction Based on A Single Image: A Review

Reconstructing A Large Scale 3D Face Dataset for Deep 3D Face Identification

Robust Geometry and Reflectance Disentanglement for 3D Face Reconstruction from Sparse-view Images

CNN-Based Real-Time Dense Face Reconstruction with Inverse-Rendered Photo-Realistic Face Images.

3D face reconstruction and dense alignment with a new generated dataset

Multi-dim: A Multi-Dimensional Face Database Towards the Application of 3D Technology in Real-World Scenarios.

Dense 3D Face Reconstruction from a Single RGB Image

Review of 3D Face Reconstruction Based on Single Image