Abstract:With the rising interest in personalized VR and gaming experiences comes the need to create high quality 3D avatars that are both low-cost and variegated. Due to this, building dynamic avatars from a single unconstrained input image is becoming a popular application. While previous techniques that attempt this require multiple input images or rely on transferring dynamic facial appearance from a source actor, we are able to do so using only one 2D input image without any form of transfer from a source image. We achieve this using a new conditional Generative Adversarial Network design that allows fine-scale manipulation of any facial input image into a new expression while preserving its identity. Our photoreal avatar GAN (paGAN) can also synthesize the unseen mouth interior and control the eye-gaze direction of the output, as well as produce the final image from a novel viewpoint. The method is even capable of generating fully-controllable temporally stable video sequences, despite not using temporal information during training. After training, we can use our network to produce dynamic image-based avatars that are controllable on mobile devices in real time. To do this, we compute a fixed set of output images that correspond to key blendshapes, from which we extract textures in UV space. Using a subject's expression blendshapes at run-time, we can linearly blend these key textures together to achieve the desired appearance. Furthermore, we can use the mouth interior and eye textures produced by our network to synthesize on-the-fly avatar animations for those regions. Our work produces state-of-the-art quality image and video synthesis, and is the first to our knowledge that is able to generate a dynamically textured avatar with a mouth interior, all from a single image.

Your3dEmoji: Creating Personalized Emojis Via One-shot 3D-Aware Cartoon Avatar Synthesis.

Generating Animatable 3D Cartoon Faces from Single Portraits

3D Cartoon Face Generation with Controllable Expressions from a Single GAN Image

StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity 3D Avatar Generation

3Dtoonify: Creating Your High-Fidelity 3D Stylized Avatar Easily from 2D Portrait Images

Synthesis, Style Editing, and Animation of 3D Cartoon Face

AgileAvatar: Stylized 3D Avatar Creation via Cascaded Domain Bridging

3D Face Cartoonizer: Generating Personalized 3D Cartoon Faces from 2D Real Photos with a Hybrid Dataset

3DFaceShop: Explicitly Controllable 3D-Aware Portrait Generation

GAN-Avatar: Controllable Personalized GAN-based Human Head Avatar

paGAN: real-time avatars using dynamic textures

AniArtAvatar: Animatable 3D Art Avatar from a Single Image

AG3D: Learning to Generate 3D Avatars from 2D Image Collections

The 3D caricature face modeling based on aesthetic formulae.

GenCA: A Text-conditioned Generative Model for Realistic and Drivable Codec Avatars

One2Avatar: Generative Implicit Head Avatar For Few-shot User Adaptation

PEGASUS: Personalized Generative 3D Avatars with Composable Attributes

Animated 3D Human Avatars from a Single Image with GAN-based Texture Inference.

Real-time Conversion from a Single 2D Face Image to a 3D Text-Driven Emotive Audio-Visual Avatar

PERSE: Personalized 3D Generative Avatars from A Single Portrait

E^3Gen: Efficient, Expressive and Editable Avatars Generation