Abstract:We present TED-Face, a new method for recovering high-fidelity 3D facial geometry and appearance with enhanced textures from single-view images. While vision-based face reconstruction has received intensive research in the past decades due to its broad applications, it remains a challenging problem because human eyes are particularly sensitive to numerically minute yet perceptually significant details. Previous methods that seek to minimize reconstruction errors within a low-dimensional face space can suffer from this issue and generate close yet low-fidelity approximations. The loss of high-frequency texture details is a key factor in their process, which we propose to address by learning to recover both dense radiance residuals and sparse facial texture features from a single image, in addition to the variables solved by previous work—shape, appearance, illumination, and camera. We integrate the estimation of all these factors in a single unified deep neural network and train it on several popular face reconstruction datasets. We also introduce two new metrics, visual fidelity (VIF) and structural similarity (SSIM), to compensate for the fact that reconstruction error is not a consistent perceptual metric of quality. On the popular FaceWarehouse facial reconstruction benchmark, our proposed system achieves a VIF score of 0.4802 and an SSIM score of 0.9622, improving over the state-of-the-art Deep3D method by 6.69% and 0.86%, respectively. On the widely used LS3D-300W dataset, we obtain a VIF score of 0.3922 and an SSIM score of 0.9079 for indoor images, and the scores for outdoor images are 0.4100 and 0.9160, respectively, which also represent an improvement over those of Deep3D. These results show that our method is able to recover visually more realistic facial appearance details compared with previous methods.

Diamond Inclusion Reconstruction

Accelerated Visual Hulls of Complex Objects Using Contribution Weights

High-fidelity 3D Face Reconstruction with Multi-Scale Details

A Convex Framework for High Resolution 3D Reconstruction.

Enhanced Narrow Band Surface Reconstruction with Anisotropic Kernel

An Algorithm to Generate Visual Hulls

Visual Hull Embossment By Graph Cuts

A High-Precision Reconstruction Algorithm of 3D Data Fields Based on Quasi-interpolation

Deep Single-View 3D Object Reconstruction with Visual Hull Embedding

Real-time Underwater 3D Reconstruction Based on Monocular Image

Incremental exact polyhedral visual hulls

Fast Polyhedral Visual Hull Reconstruction Algorithm from Photo Series

Dynamic Voxel Grid Optimization for High-Fidelity RGB-D Supervised Surface Reconstruction

A subdivision-based framework for shape reconstruction

HVOFusion: Incremental Mesh Reconstruction Using Hybrid Voxel Octree

Visual Hull Construction Using Adaptive Sampling Without Bounding Box

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

Improving RGB-D-based 3D Reconstruction by Combining Voxels and Points

VIDAR: Data Quality Improvement for Monocular 3D Reconstruction Through In-situ Visual Interaction

An Adaptive Dandelion Model for Reconstructing Spherical Terrain-Like Visual Hull Surfaces

Robust Carving for Non-Lambertian Objects