Abstract: 33D-aware face generators are commonly trained on 2D real-life face image datasets. Nevertheless, existing facial recognition methods often struggle to extract face data captured from various camera angles. Furthermore, in-the-wild images with diverse body poses introduce a high-dimensional challenge for 3D-aware generators, making it difficult to utilize data that contains complete neck and shoulder regions. Consequently, these face image datasets often contain only near-frontal face data, which poses challenges for 3D-aware face generators to construct \textit{full-head} 3D portraits. To this end, we first create the dataset {$\it{360}^{\circ}$}-\textit{Portrait}-\textit{HQ} (\textit{$\it{360}^{\circ}$PHQ}), which consists of high-quality single-view real portraits annotated with a variety of camera parameters {(the yaw angles span the entire $360^{\circ}$ range)} and body poses. We then propose \textit{3DPortraitGAN}, the first 3D-aware full-head portrait generator that learns a canonical 3D avatar distribution from the body-pose-various \textit{$\it{360}^{\circ}$PHQ} dataset with body pose self-learning. Our model can generate view-consistent portrait images from all camera angles (${360}^{\circ}$) with a full-head 3D representation. We incorporate a mesh-guided deformation field into volumetric rendering to produce deformed results to generate portrait images that conform to the body pose distribution of the dataset using our canonical generator. We integrate two pose predictors into our framework to predict more accurate body poses to address the issue of inaccurately estimated body poses in our dataset. Our experiments show that the proposed framework can generate view-consistent, realistic portrait images with complete geometry from all camera angles and accurately predict portrait body pose.

Learning to Generate 3D-Aware Realistic Hand from 2D and 3D Priors

Synthesizing Depth Hand Images with GANs and Style Transfer for Hand Pose Estimation

XHand: Real-time Expressive Hand Avatar

Learning Interaction-aware 3D Gaussian Splatting for One-shot Hand Avatars

RealisticHands: A Hybrid Model for 3D Hand Reconstruction

Generating Realistic Training Images Based on Tonality-Alignment Generative Adversarial Networks for Hand Pose Estimation

AG3D: Learning to Generate 3D Avatars from 2D Image Collections

Deformation Representation Based Convolutional Mesh Autoencoder for 3D Hand Generation

FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generation

DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth

Get3DHuman: Lifting StyleGAN-Human into a 3D Generative Model using Pixel-aligned Reconstruction Priors

Body2Hands: Learning to Infer 3D Hands from Conversational Gesture Body Dynamics

Hand3D: Hand Pose Estimation using 3D Neural Network

Annotated Hands for Generative Models

AttentionHand: Text-driven Controllable Hand Image Generation for 3D Hand Reconstruction in the Wild

Improvements in 3D Hand Pose Estimation Using Synthetic Data

Next3D: Generative Neural Texture Rasterization for 3D-Aware Head Avatars

Giving a Hand to Diffusion Models: a Two-Stage Approach to Improving Conditional Human Image Generation

GANHead: Towards Generative Animatable Neural Head Avatars

Learning Full-Head 3D GANs from a Single-View Portrait Dataset

3D-Aware Semantic-Guided Generative Model for Human Synthesis.