Abstract:Although learning‐based 3D face reconstruction methods have exhibited outstanding performance, their effectiveness is severely constrained by the scarcity of available training data with 3D annotations. To address this issue, we present the PR3D (Precise and Realistic 3D face reconstruction) method, which consists of high‐precision shape reconstruction based on semi‐supervised learning and high‐fidelity texture reconstruction based on StyleGAN2. Furthermore, to meet the real‐time requirements in practical applications, a lightweight shape reconstruction model called fast‐PR3D is distilled through teacher‐student learning. Reconstructing the three‐dimensional (3D) shape and texture of the face from a single image is a significant and challenging task in computer vision and graphics. In recent years, learning‐based reconstruction methods have exhibited outstanding performance, but their effectiveness is severely constrained by the scarcity of available training data with 3D annotations. To address this issue, we present the PR3D (Precise and Realistic 3D face reconstruction) method, which consists of high‐precision shape reconstruction based on semi‐supervised learning and high‐fidelity texture reconstruction based on StyleGAN2. In shape reconstruction, we use in‐the‐wild face images and 3D annotated datasets to train the auxiliary encoder and the identity encoder, encoding the input image into parameters of FLAME (a parametric 3D face model). Simultaneously, a novel semi‐supervised hybrid landmark loss is designed to more effectively learn from in‐the‐wild face images and 3D annotated datasets. Furthermore, to meet the real‐time requirements in practical applications, a lightweight shape reconstruction model called fast‐PR3D is distilled through teacher–student learning. In texture reconstruction, we propose a texture extraction method based on face reenactment in StyleGAN2 style space, extracting texture from the source and reenacted face images to constitute a facial texture map. Extensive experiments have demonstrated the state‐of‐the‐art performance of our method.

PR3D: Precise and realistic 3D face reconstruction from a single image

A synthesis method for personalized 3D face reconstruction

High-fidelity 3D Face Reconstruction with Multi-Scale Details

Realistic Reconstruction of Human Face Based on Images

“Edutainment 2017” a Visual and Semantic Representation of 3D Face Model for Reshaping Face in Images

New Algorithm for 3d Facial Model Reconstruction and Its Application in Virtual Reality

Unsupervised Style-based Explicit 3D Face Reconstruction from Single Image

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

Learning to Restore 3D Face from In-the-Wild Degraded Images

Texture-Shape Optimized GAT for 3D Face Reconstruction.

3D Face Reconstruction and Semantic Annotation from Single Depth Image

Research on 3D Face Reconstruction Algorithm Based on ResNet and Transformer

Realistic 3D Face Modeling by Fusing Multiple 2D Images

Rendering with style

Face Denoising and 3D Reconstruction from A Single Depth Image

VGG-Tex: A Vivid Geometry-Guided Facial Texture Estimation Model for High Fidelity Monocular 3D Face Reconstruction

Accurate 3D Face Reconstruction With Weakly-Supervised Learning: From Single Image to Image Set

Dynamic 3D Facial Expression Reconstruction from Images

A New Algorithm for 3d Facial Model Reconstruction and Its Application in Vr

3D Face Reconstruction Based on A Single Image: A Review

3D Face Reconstruction Based on ResNet Feature Extraction and CBAM