Abstract:Text-to-image generation intends to automatically produce a photo-realistic image, conditioned on a textual description. It can be potentially employed in the field of art creation, data augmentation, photo-editing, etc. Although many efforts have been dedicated to this task, it remains particularly challenging to generate believable, natural scenes. To facilitate the real-world applications of text-to-image synthesis, we focus on studying the following three issues: 1) How to ensure that generated samples are believable, realistic or natural? 2) How to exploit the latent space of the generator to edit a synthesized image? 3) How to improve the explainability of a text-to-image generation framework? In this work, we constructed two novel data sets (i.e., the Good & Bad bird and face data sets) consisting of successful as well as unsuccessful generated samples, according to strict criteria. To effectively and efficiently acquire high-quality images by increasing the probability of generating Good latent codes, we use a dedicated Good/Bad classifier for generated images. It is based on a pre-trained front end and fine-tuned on the basis of the proposed Good & Bad data set. After that, we present a novel algorithm which identifies semantically-understandable directions in the latent space of a conditional text-to-image GAN architecture by performing independent component analysis on the pre-trained weight values of the generator. Furthermore, we develop a background-flattening loss (BFL), to improve the background appearance in the edited image. Subsequently, we introduce linear interpolation analysis between pairs of keywords. This is extended into a similar triangular `linguistic' interpolation in order to take a deep look into what a text-to-image synthesis model has learned within the linguistic embeddings. Our data set is available at <a class="link-external link-https" href="https://zenodo.org/record/6283798#.YhkN_ujMI2w" rel="external noopener nofollow">this https URL</a>.

RenderGAN: GAN Based Texture Rendering

3D-Aware Image Synthesis Via Learning Structural and Textural Representations

SpatialGAN: Progressive Image Generation Based on Spatial Recursive Adversarial Expansion

GramGAN: Deep 3D Texture Synthesis From 2D Exemplars

Efficient texture-aware multi-GAN for image inpainting

Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis.

Two Birds with One Stone: Iteratively Learn Facial Attributes with GANs.

Texture Representation via Analysis and Synthesis with Generative Adversarial Networks

What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs

Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering

Progressively Unfreezing Perceptual GAN

Fine Detailed Texture Learning for 3D Meshes with Generative Models

Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Data

Pixel art character generation as an image-to-image translation problem using GANs

OptGAN: Optimizing and Interpreting the Latent Space of the Conditional Text-to-Image GANs

CTGAN: Semantic-guided Conditional Texture Generator for 3D Shapes

STS-GAN: Can We Synthesize Solid Texture with High Fidelity from Arbitrary 2D Exemplar?

GSGAN: Adversarial Learning for Hierarchical Generation of 3D Gaussian Splats

ContraNeRF: 3D-Aware Generative Model via Contrastive Learning with Unsupervised Implicit Pose Embedding

DAM-GAN : Image Inpainting using Dynamic Attention Map based on Fake Texture Detection

Portrait3D: Text-Guided High-Quality 3D Portrait Generation Using Pyramid Representation and GANs Prior