Abstract:The existing text-guided image synthesis methods can only produce limited quality results with at most \mbox{$\text{256}^2$} resolution and the textual instructions are constrained in a small Corpus. In this work, we propose a unified framework for both face image generation and manipulation that produces diverse and high-quality images with an unprecedented resolution at 1024 from multimodal inputs. More importantly, our method supports open-world scenarios, including both image and text, without any re-training, fine-tuning, or post-processing. To be specific, we propose a brand new paradigm of text-guided image generation and manipulation based on the superior characteristics of a pretrained GAN model. Our proposed paradigm includes two novel strategies. The first strategy is to train a text encoder to obtain latent codes that align with the hierarchically semantic of the aforementioned pretrained GAN model. The second strategy is to directly optimize the latent codes in the latent space of the pretrained GAN model with guidance from a pretrained language model. The latent codes can be randomly sampled from a prior distribution or inverted from a given image, which provides inherent supports for both image generation and manipulation from multi-modal inputs, such as sketches or semantic labels, with textual guidance. To facilitate text-guided multi-modal synthesis, we propose the Multi-Modal CelebA-HQ, a large-scale dataset consisting of real face images and corresponding semantic segmentation map, sketch, and textual descriptions. Extensive experiments on the introduced dataset demonstrate the superior performance of our proposed method. Code and data are available at <a class="link-external link-https" href="https://github.com/weihaox/TediGAN" rel="external noopener nofollow">this https URL</a>.

An improved StyleGAN-based TextToFace model with Local-Global information Fusion

DualG-GAN, a Dual-channel Generator based Generative Adversarial Network for text-to-face synthesis

A face template: Improving the face generation quality of multi-stage generative adversarial networks using coarse-grained facial priors

Generating Distinctive Facial Images from Natural Language Descriptions Via Spatial Map Fusion

Text2FaceGAN: Face Generation from Fine Grained Textual Descriptions

TediGAN: Text-Guided Diverse Face Image Generation and Manipulation

StyleIPSB: Identity-Preserving Semantic Basis of StyleGAN for High Fidelity Face Swapping

ISFB-GAN: Interpretable semantic face beautification with generative adversarial network

FacialGAN: Style Transfer and Attribute Manipulation on Synthetic Faces

Controllable 3D Face Generation with Conditional Style Code Diffusion

Multi-Modal Face Stylization with a Generative Prior

Text-to-Face Generation with StyleGAN2

High-Fidelity Face Swapping with Style Blending

Local and Global Perception Generative Adversarial Network for Facial Expression Synthesis

CMAFGAN: A Cross-Modal Attention Fusion based Generative Adversarial Network for attribute word-to-face synthesis

Towards Open-World Text-Guided Face Image Generation and Manipulation

DMF-GAN: Deep Multimodal Fusion Generative Adversarial Networks for Text-to-Image Synthesis

Specialized discriminators for style consistency in facial expression synthesis

VoiceStyle: Voice-based Face Generation Via Cross-modal Prototype Contrastive Learning

Towards Spatially Disentangled Manipulation of Face Images With Pre-Trained StyleGANs

Unconstrained Facial Expression Transfer using Style-based Generator