Abstract:We present TED-Face, a new method for recovering high-fidelity 3D facial geometry and appearance with enhanced textures from single-view images. While vision-based face reconstruction has received intensive research in the past decades due to its broad applications, it remains a challenging problem because human eyes are particularly sensitive to numerically minute yet perceptually significant details. Previous methods that seek to minimize reconstruction errors within a low-dimensional face space can suffer from this issue and generate close yet low-fidelity approximations. The loss of high-frequency texture details is a key factor in their process, which we propose to address by learning to recover both dense radiance residuals and sparse facial texture features from a single image, in addition to the variables solved by previous work—shape, appearance, illumination, and camera. We integrate the estimation of all these factors in a single unified deep neural network and train it on several popular face reconstruction datasets. We also introduce two new metrics, visual fidelity (VIF) and structural similarity (SSIM), to compensate for the fact that reconstruction error is not a consistent perceptual metric of quality. On the popular FaceWarehouse facial reconstruction benchmark, our proposed system achieves a VIF score of 0.4802 and an SSIM score of 0.9622, improving over the state-of-the-art Deep3D method by 6.69% and 0.86%, respectively. On the widely used LS3D-300W dataset, we obtain a VIF score of 0.3922 and an SSIM score of 0.9079 for indoor images, and the scores for outdoor images are 0.4100 and 0.9160, respectively, which also represent an improvement over those of Deep3D. These results show that our method is able to recover visually more realistic facial appearance details compared with previous methods.

3D Face Reconstruction with Feature Enhancement using Bi-FPN for Forensic Analysis

High-fidelity 3D Face Reconstruction with Multi-Scale Details

3D Face Reconstruction Techniques: Passive Methods

Geometry-aware Two-scale PIFu Representation for Human Reconstruction

Dynamic 3D Facial Expression Reconstruction from Images

3D Face Reconstruction From Single 2D Image Using Distinctive Features

A proposed reconstruction method of a 3D animation scene based on a fuzzy long and short-term memory algorithm

Joint 3d Face Reconstruction And Dense Face Alignment Via Deep Face Feature Alignment

3D Face Reconstruction: the Road to Forensics

Extreme 3D Face Reconstruction: Seeing Through Occlusions

Fine-Scale Face Fitting and Texture Fusion With Inverse Renderer

3DFaceNet: Real-time Dense Face Reconstruction via Synthesizing Photo-realistic Face Images

A Hierarchical Representation Network for Accurate and Detailed Face Reconstruction from In-The-Wild Images

Skull-to-Face: Anatomy-Guided 3D Facial Reconstruction and Editing

3d Face Reconstruction With Texture Details From A Single Image Based On Gan

DEEP TEXTURE FEATURES FOR ROBUST FACE SPOOFING DETECTION

3D Face Reconstruction for Forensic Recognition -- A Survey

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

CNN-Based Real-Time Dense Face Reconstruction with Inverse-Rendered Photo-Realistic Face Images.

3D face reconstruction from mugshots: Application to arbitrary view face recognition

A Face Reconstruction Method Based On Fusion Regression Network And Gradient Descent