Abstract:We present TED-Face, a new method for recovering high-fidelity 3D facial geometry and appearance with enhanced textures from single-view images. While vision-based face reconstruction has received intensive research in the past decades due to its broad applications, it remains a challenging problem because human eyes are particularly sensitive to numerically minute yet perceptually significant details. Previous methods that seek to minimize reconstruction errors within a low-dimensional face space can suffer from this issue and generate close yet low-fidelity approximations. The loss of high-frequency texture details is a key factor in their process, which we propose to address by learning to recover both dense radiance residuals and sparse facial texture features from a single image, in addition to the variables solved by previous work—shape, appearance, illumination, and camera. We integrate the estimation of all these factors in a single unified deep neural network and train it on several popular face reconstruction datasets. We also introduce two new metrics, visual fidelity (VIF) and structural similarity (SSIM), to compensate for the fact that reconstruction error is not a consistent perceptual metric of quality. On the popular FaceWarehouse facial reconstruction benchmark, our proposed system achieves a VIF score of 0.4802 and an SSIM score of 0.9622, improving over the state-of-the-art Deep3D method by 6.69% and 0.86%, respectively. On the widely used LS3D-300W dataset, we obtain a VIF score of 0.3922 and an SSIM score of 0.9079 for indoor images, and the scores for outdoor images are 0.4100 and 0.9160, respectively, which also represent an improvement over those of Deep3D. These results show that our method is able to recover visually more realistic facial appearance details compared with previous methods.

3D Face Reconstruction and Semantic Annotation from Single Depth Image

Robust 3d Face Reconstruction From Single Noisy Depth Image Through Semantic Consistency

High-fidelity 3D Face Reconstruction with Multi-Scale Details

Face Denoising and 3D Reconstruction from A Single Depth Image

Semantic Reconstruction based on RGB Image and Sparse Depth

“Edutainment 2017” a Visual and Semantic Representation of 3D Face Model for Reshaping Face in Images

Depth-Based 3D Face Reconstruction and Pose Estimation Using Shape-Preserving Domain Adaptation

Learning Semantic Representations via Joint 3D Face Reconstruction and Facial Attribute Estimation

FaceScape: 3D Facial Dataset and Benchmark for Single-View 3D Face Reconstruction

Feature Sharing Attention 3D Face Reconstruction with Unsupervised Learning from In-the-Wild Photo Collection

3D Face Reconstruction with Geometry Details from a Single Image

Dense 3D Facial Reconstruction from a Single Depth Image in Unconstrained Environment

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

Beyond 3DMM Space: Towards Fine-Grained 3D Face Reconstruction

3DFaceNet: Real-time Dense Face Reconstruction via Synthesizing Photo-realistic Face Images

3D Face Reconstruction Using A Spectral-Based Graph Convolution Encoder

Dense 3D Face Reconstruction from a Single RGB Image

Sparse Photometric 3D Face Reconstruction Guided by Morphable Models

Fine-grained 3D Face Reconstruction from a Single Image Using Illumination Priors

3D Face Reconstruction System Based on Deep Learning and Sparse Face Model

Single Image Based Three-Dimensional Scene Reconstruction Using Semantic and Geometric Priors