Abstract:We present TED-Face, a new method for recovering high-fidelity 3D facial geometry and appearance with enhanced textures from single-view images. While vision-based face reconstruction has received intensive research in the past decades due to its broad applications, it remains a challenging problem because human eyes are particularly sensitive to numerically minute yet perceptually significant details. Previous methods that seek to minimize reconstruction errors within a low-dimensional face space can suffer from this issue and generate close yet low-fidelity approximations. The loss of high-frequency texture details is a key factor in their process, which we propose to address by learning to recover both dense radiance residuals and sparse facial texture features from a single image, in addition to the variables solved by previous work—shape, appearance, illumination, and camera. We integrate the estimation of all these factors in a single unified deep neural network and train it on several popular face reconstruction datasets. We also introduce two new metrics, visual fidelity (VIF) and structural similarity (SSIM), to compensate for the fact that reconstruction error is not a consistent perceptual metric of quality. On the popular FaceWarehouse facial reconstruction benchmark, our proposed system achieves a VIF score of 0.4802 and an SSIM score of 0.9622, improving over the state-of-the-art Deep3D method by 6.69% and 0.86%, respectively. On the widely used LS3D-300W dataset, we obtain a VIF score of 0.3922 and an SSIM score of 0.9079 for indoor images, and the scores for outdoor images are 0.4100 and 0.9160, respectively, which also represent an improvement over those of Deep3D. These results show that our method is able to recover visually more realistic facial appearance details compared with previous methods.

High-fidelity 3D Face Reconstruction with Multi-Scale Details

A synthesis method for personalized 3D face reconstruction

HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Details

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

Geometry-aware Two-scale PIFu Representation for Human Reconstruction

3D Face Reconstruction with Geometry Details from a Single Image

Realistic Reconstruction of Human Face Based on Images

Coarse-to-fine Multiview 3d Face Reconstruction Using Multiple Geometrical Features.

Facial Geometric Detail Recovery via Implicit Representation

3D Face Reconstruction with Feature Enhancement using Bi-FPN for Forensic Analysis

Towards High-Fidelity 3D Face Reconstruction from In-the-Wild Images Using Graph Convolutional Networks

Realistic 3D Face Modeling by Fusing Multiple 2D Images

High-Fidelity Facial Albedo Estimation via Texture Quantization

Robust Geometry and Reflectance Disentanglement for 3D Face Reconstruction from Sparse-view Images

Single-view 3D Scene Reconstruction with High-fidelity Shape and Texture

High Fidelity 3D Hand Shape Reconstruction via Scalable Graph Frequency Decomposition

3D Face Reconstruction and Semantic Annotation from Single Depth Image

Extreme 3D Face Reconstruction: Seeing Through Occlusions

ReFu: Refine and Fuse the Unobserved View for Detail-Preserving Single-Image 3D Human Reconstruction

Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set

FineRecon: Depth-aware Feed-forward Network for Detailed 3D Reconstruction