Abstract:High-fidelity face digitization solutions often combine multi-view stereo (MVS) techniques for 3D reconstruction and a non-rigid registration step to establish dense correspondence across identities and expressions. A common problem is the need for manual clean-up after the MVS step, as 3D scans are typically affected by noise and outliers and contain hairy surface regions that need to be cleaned up by artists. Furthermore, mesh registration tends to fail for extreme facial expressions. Most learning-based methods use an underlying 3D morphable model (3DMM) to ensure robustness, but this limits the output accuracy for extreme facial expressions. In addition, the global bottleneck of regression architectures cannot produce meshes that tightly fit the ground truth surfaces. We propose ToFu, Topologically consistent Face from multi-view, a geometry inference framework that can produce topologically consistent meshes across facial identities and expressions using a volumetric representation instead of an explicit underlying 3DMM. Our novel progressive mesh generation network embeds the topological structure of the face in a feature volume, sampled from geometry-aware local features. A coarse-to-fine architecture facilitates dense and accurate facial mesh predictions in a consistent mesh topology. ToFu further captures displacement maps for pore-level geometric details and facilitates high-quality rendering in the form of albedo and specular reflectance maps. These high-quality assets are readily usable by production studios for avatar creation, animation and physically-based skin rendering. We demonstrate state-of-the-art geometric and correspondence accuracy, while only taking 0.385 seconds to compute a mesh with 10K vertices, which is three orders of magnitude faster than traditional techniques. The code and the model are available for research purposes at <a class="link-external link-https" href="https://tianyeli.github.io/tofu" rel="external noopener nofollow">this https URL</a>.

SR-VFA: Accurate Self-Refined Face Alignment in Videos.

A synthesis method for personalized 3D face reconstruction

Face Alignment Based on 3D Face Shape Model and Markov Random Field.

Face alignment based on high order markov random field

High-fidelity 3D Face Reconstruction with Multi-Scale Details

Towards Fast, Accurate and Stable 3D Dense Face Alignment

Automatic 3D Face Modeling Using 2D Active Appearance Models

Coarse-to-fine Multiview 3d Face Reconstruction Using Multiple Geometrical Features.

Unified Video and Image Representation for Boosted Video Face Forgery Detection

RAFaRe: Learning Robust and Accurate Non-parametric 3D Face Reconstruction from Pseudo 2D&3D Pairs

Multi-view face alignment using 3D shape model for view estimation

Video Tracked Facial Expression Animation

Automatic 3D Face Modeling from Video

An Automatic Method of Building 3D Morphable Face Model

Refined dense face alignment through image matching

Real-Time Face Alignment With Tracking In Video

Topologically Consistent Multi-View Face Inference Using Volumetric Sampling

Efficient Video Face Enhancement with Enhanced Spatial-Temporal Consistency

Realistic 3D Face Modeling by Fusing Multiple 2D Images

MMFace: A Multi-Metric Regression Network for Unconstrained Face Reconstruction

Enhancing Facial Consistency in Conditional Video Generation via Facial Landmark Transformation