Abstract:Although learning‐based 3D face reconstruction methods have exhibited outstanding performance, their effectiveness is severely constrained by the scarcity of available training data with 3D annotations. To address this issue, we present the PR3D (Precise and Realistic 3D face reconstruction) method, which consists of high‐precision shape reconstruction based on semi‐supervised learning and high‐fidelity texture reconstruction based on StyleGAN2. Furthermore, to meet the real‐time requirements in practical applications, a lightweight shape reconstruction model called fast‐PR3D is distilled through teacher‐student learning. Reconstructing the three‐dimensional (3D) shape and texture of the face from a single image is a significant and challenging task in computer vision and graphics. In recent years, learning‐based reconstruction methods have exhibited outstanding performance, but their effectiveness is severely constrained by the scarcity of available training data with 3D annotations. To address this issue, we present the PR3D (Precise and Realistic 3D face reconstruction) method, which consists of high‐precision shape reconstruction based on semi‐supervised learning and high‐fidelity texture reconstruction based on StyleGAN2. In shape reconstruction, we use in‐the‐wild face images and 3D annotated datasets to train the auxiliary encoder and the identity encoder, encoding the input image into parameters of FLAME (a parametric 3D face model). Simultaneously, a novel semi‐supervised hybrid landmark loss is designed to more effectively learn from in‐the‐wild face images and 3D annotated datasets. Furthermore, to meet the real‐time requirements in practical applications, a lightweight shape reconstruction model called fast‐PR3D is distilled through teacher–student learning. In texture reconstruction, we propose a texture extraction method based on face reenactment in StyleGAN2 style space, extracting texture from the source and reenacted face images to constitute a facial texture map. Extensive experiments have demonstrated the state‐of‐the‐art performance of our method.

Texture-Shape Optimized GAT for 3D Face Reconstruction.

A Cross-Dimension Annotations Method for 3D Structural Facial Landmark Extraction

High-fidelity 3D Face Reconstruction with Multi-Scale Details

3D-GANTex: 3D Face Reconstruction with StyleGAN3-based Multi-View Images and 3DDFA based Mesh Generation

VGG-Tex: A Vivid Geometry-Guided Facial Texture Estimation Model for High Fidelity Monocular 3D Face Reconstruction

PR3D: Precise and realistic 3D face reconstruction from a single image

Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction

GANFIT: Generative Adversarial Network Fitting for High Fidelity 3D Face Reconstruction

MPF-GAN: an enhanced architecture for 3D face reconstruction

Two Birds with One Stone: Iteratively Learn Facial Attributes with GANs.

Towards High-Fidelity 3D Face Reconstruction from In-the-Wild Images Using Graph Convolutional Networks

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild

Faster, Better and More Detailed: 3D Face Reconstruction with Graph Convolutional Networks

Progressive Learning of 3D Reconstruction Network from 2D GAN Data

3D-Mask-GAN:Unsupervised Single-View 3D Object Reconstruction

Advanced 3D Face Reconstruction from Single 2D Images Using Enhanced Adversarial Neural Networks and Graph Neural Networks

Joint 3D facial shape reconstruction and texture completion from a single image

AgileGAN3D: Few-Shot 3D Portrait Stylization by Augmented Transfer Learning

Unsupervised Style-based Explicit 3D Face Reconstruction from Single Image

AvatarMe++: Facial Shape and BRDF Inference With Photorealistic Rendering-Aware GANs

3D Face Reconstruction Based on ResNet Feature Extraction and CBAM