Abstract:Various applications require realistic, artifact-free, and animatable 3D avatars. However, traditional 3D morphable models (3DMMs) produce animatable 3D heads but fail to capture accurate geometries and details, while existing deep implicit functions have been shown to achieve realistic reconstructions but suffer from artifacts and struggle to yield 3D heads that are easy to animate. To reconstruct high-fidelity, artifact-less, and animatable 3D heads from single-view images, we leverage semantics to bridge the best properties of 3DMMs and deep implicit functions and propose SeIF—a semantic-constrained deep implicit function. First, SeIF derives fine-grained semantics from a standard 3DMM (e.g., FLAME) and samples a semantic code for each query point in the query space to provide a soft constraint to the deep implicit function. The reconstruction results show that this semantic constraint does not weaken the powerful representation ability of the deep implicit function while significantly suppressing artifacts. Second, SeIF predicts a more accurate semantic code for each query point and utilizes the semantic codes to uniformize the structure of reconstructed 3D head meshes with the standard 3DMM. Since our reconstructed 3D head meshes have the same structure as the 3DMM, 3DMM-based animation approaches can be easily transferred to animate our reconstructed 3D heads. As a result, SeIF can reconstruct high-fidelity, artifact-less, and animatable 3D heads from single-view images of individuals with diverse ages, genders, races, and facial expressions. Quantitative and qualitative experimental results on seven datasets show that SeIF outperforms existing state-of-the-art methods by a large margin.

SeIF: Semantic-Constrained Deep Implicit Function for Single-Image 3D Head Reconstruction

SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Video

3D Face Reconstruction and Semantic Annotation from Single Depth Image

TIFu: Tri-directional Implicit Function for High-Fidelity 3D Character Reconstruction

SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstruction

Multi‐Level Implicit Function for Detailed Human Reconstruction by Relaxing SMPL Constraints

Learning Pose Controllable Human Reconstruction with Dynamic Implicit Fields from a Single Image

Implicit Neural Head Synthesis via Controllable Local Deformation Fields

ID-to-3D: Expressive ID-guided 3D Heads via Score Distillation Sampling

Visual and Spatial Context Fusion for Implicit Human Reconstruction

UNIF: United Neural Implicit Functions for Clothed Human Reconstruction and Animation

ImFace++: A Sophisticated Nonlinear 3D Morphable Face Model with Implicit Neural Representations

View Consistent 3D Face Reconstruction Using Siamese Encoder-Decoders

HeadRecon: High-Fidelity 3D Head Reconstruction from Monocular Video

Realistic 3D Face Modeling by Fusing Multiple 2D Images

High-Fidelity 3D Head Avatars Reconstruction through Spatially-Varying Expression Conditioned Neural Radiance Field

OmniAvatar: Geometry-Guided Controllable 3D Head Synthesis

SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion

3D3M: 3D Modulated Morphable Model for Monocular Face Reconstruction

SRIF: Semantic Shape Registration Empowered by Diffusion-based Image Morphing and Flow Estimation