Abstract: 33D-aware face generators are commonly trained on 2D real-life face image datasets. Nevertheless, existing facial recognition methods often struggle to extract face data captured from various camera angles. Furthermore, in-the-wild images with diverse body poses introduce a high-dimensional challenge for 3D-aware generators, making it difficult to utilize data that contains complete neck and shoulder regions. Consequently, these face image datasets often contain only near-frontal face data, which poses challenges for 3D-aware face generators to construct \textit{full-head} 3D portraits. To this end, we first create the dataset {$\it{360}^{\circ}$}-\textit{Portrait}-\textit{HQ} (\textit{$\it{360}^{\circ}$PHQ}), which consists of high-quality single-view real portraits annotated with a variety of camera parameters {(the yaw angles span the entire $360^{\circ}$ range)} and body poses. We then propose \textit{3DPortraitGAN}, the first 3D-aware full-head portrait generator that learns a canonical 3D avatar distribution from the body-pose-various \textit{$\it{360}^{\circ}$PHQ} dataset with body pose self-learning. Our model can generate view-consistent portrait images from all camera angles (${360}^{\circ}$) with a full-head 3D representation. We incorporate a mesh-guided deformation field into volumetric rendering to produce deformed results to generate portrait images that conform to the body pose distribution of the dataset using our canonical generator. We integrate two pose predictors into our framework to predict more accurate body poses to address the issue of inaccurately estimated body poses in our dataset. Our experiments show that the proposed framework can generate view-consistent, realistic portrait images with complete geometry from all camera angles and accurately predict portrait body pose.

3D-Mask-GAN:Unsupervised Single-View 3D Object Reconstruction

2D GANs Meet Unsupervised Single-view 3D Reconstruction

Semi-supervised Three-dimensional Reconstruction Framework with GAN.

Self-supervised single-view 3D point cloud reconstruction through GAN inversion

Progressive Learning of 3D Reconstruction Network from 2D GAN Data

Monocular 3D Object Reconstruction with GAN Inversion

Do 2D GANs Know 3D Shape? Unsupervised 3D shape reconstruction from 2D Image GANs

Unsupervised Style-based Explicit 3D Face Reconstruction from Single Image

Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction

Shape-from-Mask: A Deep Learning Based Human Body Shape Reconstruction from Binary Mask Images

Deep Single-View 3D Object Reconstruction with Visual Hull Embedding

Get3DHuman: Lifting StyleGAN-Human into a 3D Generative Model using Pixel-aligned Reconstruction Priors

Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction

Single-view 3D Mesh Reconstruction for Seen and Unseen Categories

Learning Full-Head 3D GANs from a Single-View Portrait Dataset

Fast-GANFIT: Generative Adversarial Network for High Fidelity 3D Face Reconstruction

Unsupervised Learning of Depth Estimation and Camera Pose With Multi-Scale GANs

3D generation on ImageNet

SG-GAN: Fine Stereoscopic-Aware Generation for 3D Brain Point Cloud Up-sampling from a Single Image

3DPortraitGAN: Learning One-Quarter Headshot 3D GANs from a Single-View Portrait Dataset with Diverse Body Poses