Abstract:Generating high-quality and diverse human images is an important yet challenging task in vision and graphics. However, existing generative models often fall short under the high diversity of clothing shapes and textures. Furthermore, the generation process is even desired to be intuitively controllable for layman users. In this work, we present a text-driven controllable framework, Text2Human, for a high-quality and diverse human generation. We synthesize full-body human images starting from a given human pose with two dedicated steps. 1) With some texts describing the shapes of clothes, the given human pose is first translated to a human parsing map. 2) The final human image is then generated by providing the system with more attributes about the textures of clothes. Specifically, to model the diversity of clothing textures, we build a hierarchical texture-aware codebook that stores multi-scale neural representations for each type of texture. The codebook at the coarse level includes the structural representations of textures, while the codebook at the fine level focuses on the details of textures. To make use of the learned hierarchical codebook to synthesize desired images, a diffusion-based transformer sampler with mixture of experts is firstly employed to sample indices from the coarsest level of the codebook, which then is used to predict the indices of the codebook at finer levels. The predicted indices at different levels are translated to human images by the decoder learned accompanied with hierarchical codebooks. The use of mixture-of-experts allows for the generated image conditioned on the fine-grained text input. The prediction for finer level indices refines the quality of clothing textures. Extensive quantitative and qualitative evaluations demonstrate that our proposed framework can generate more diverse and realistic human images compared to state-of-the-art methods.

UVMap-ID: A Controllable and Personalized UV Map Generative Model

Towards high-fidelity facial UV map generation in real-world

Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models

UV-IDM: Identity-Conditioned Latent Diffusion Model for Face UV-Texture Generation

TEXGen: a Generative Diffusion Model for Mesh Textures

Texture Generation on 3D Meshes with Point-UV Diffusion

TUVF: Learning Generalizable Texture UV Radiance Fields

UV-free Texture Generation with Denoising and Geodesic Heat Diffusions

UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition

Subject-Diffusion:Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning

FaceChain: A Playground for Identity-Preserving Portrait Generation

IFaceUV: Intuitive Motion Facial Image Generation by Identity Preservation via UV map

TexDreamer: Towards Zero-Shot High-Fidelity 3D Human Texture Generation

TexGen: Text-Guided 3D Texture Generation with Multi-view Sampling and Resampling

Text2Human: Text-Driven Controllable Human Image Generation.

Magic-Me: Identity-Specific Video Customized Diffusion

U-VAP: User-specified Visual Appearance Personalization via Decoupled Self Augmentation

UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with Authenticity Guided Textures

MVTexGen: Synthesising 3D Textures Using Multi-View Diffusion

Make-It-Vivid: Dressing Your Animatable Biped Cartoon Characters from Text

Cloth2Tex: A Customized Cloth Texture Generation Pipeline for 3D Virtual Try-On