Abstract:Reconstructing high-fidelity 3D facial texture from a single image is a quite challenging task due to the lack of complete face information and the domain gap between the 3D face and 2D image. Further, obtaining re-renderable 3D faces has become a strongly desired property in many applications, where the term 're-renderable' demands the facial texture to be spatially complete and disentangled with environmental illumination. In this paper, we propose a new self-supervised deep learning framework for reconstructing high-quality and re-renderable facial albedos from single-view images in-the-wild. Our main idea is to first utilize a prior generation module based on the 3DMM proxy model to produce an unwrapped texture and a globally parameterized prior albedo. Then we apply a detail refinement module to synthesize the final texture with both high-frequency details and completeness. To further make facial textures disentangled with illumination, we propose a novel detailed illumination representation which is reconstructed with the detailed albedo together. We also design several novel regularization losses on both the albedo and illumination maps to facilitate the disentanglement of these two factors. Finally, by leveraging a differentiable renderer, each face attribute can be jointly trained in a self-supervised manner without requiring ground-truth facial reflectance. Extensive comparisons and ablation studies on challenging datasets demonstrate that our framework outperforms state-of-the-art approaches.

Robust and Detailed Reconstuction of Human Face from Single Image

A synthesis method for personalized 3D face reconstruction

High-fidelity 3D Face Reconstruction with Multi-Scale Details

Realistic Reconstruction of Human Face Based on Images

MMFace: A Multi-Metric Regression Network for Unconstrained Face Reconstruction

3D3M: 3D Modulated Morphable Model for Monocular Face Reconstruction

3D Face Reconstruction with Geometry Details from a Single Image

Learning-based Fully 3D Face Reconstruction from a Single Image

Robust Geometry and Reflectance Disentanglement for 3D Face Reconstruction from Sparse-view Images

Self-Supervised 3d Face Reconstruction Based On Multi-View Uv Fusion

Towards High-Fidelity 3D Face Reconstruction from In-the-Wild Images Using Graph Convolutional Networks

Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set

Detailed 3D Face Reconstruction from Single Images Via Self-supervised Attribute Learning

Robust 3d Face Reconstruction From Single Noisy Depth Image Through Semantic Consistency

Self-supervised Re-renderable Facial Albedo Reconstruction from Single Image

Beyond 3DMM Space: Towards Fine-Grained 3D Face Reconstruction

HiFace: High-Fidelity 3D Face Reconstruction by Learning Static and Dynamic Details

Realistic 3D Face Modeling by Fusing Multiple 2D Images

Multi-pose 3D Facial Texture Refinement for Face Recognition.

Extreme 3D Face Reconstruction: Seeing Through Occlusions

3D Face Reconstruction from One Image Based on Generic Model