Abstract:Text-to-image synthesis refers to generating an image from a given text description, the key goal of which lies in photo realism and semantic consistency. Previous methods usually generate an initial image with sentence embedding and then refine it with fine-grained word embedding. Despite the significant progress, the 'aspect' information (e.g., red eyes) contained in the text, referring to several words rather than a word that depicts `a particular part or feature of something', is often ignored, which is highly helpful for synthesizing image details. How to make better utilization of aspect information in text-to-image synthesis still remains an unresolved challenge. To address this problem, in this paper, we propose a Dynamic Aspect-awarE GAN (DAE-GAN) that represents text information comprehensively from multiple granularities, including sentencelevel, word-level, and aspect-level. Moreover, inspired by human learning behaviors, we develop a novel Aspectaware Dynamic Re-drawer (ADR) for image refinement, in which an Attended Global Refinement (AGR) module and an Aspect-aware Local Refinement (ALR) module are alternately employed. AGR utilizes word-level embedding to globally enhance the previously generated image, while ALR dynamically employs aspect-level embedding to refine image details from a local perspective. Finally, a corresponding matching loss function is designed to ensure the text-image semantic consistency at different levels. Extensive experiments on two well-studied and publicly available datasets (i.e., CUB-200 and COCO) demonstrate the superiority and rationality of our method.

Enriching Attributes from Knowledge Graph for Fine-grained Text-to-Image Synthesis.

Fine-grained Semantic Constraint in Image Synthesis

Attribute-Guided Sketch Generation

KnHiGAN: Knowledge-enhanced Hierarchical Generative Adversarial Network for Fine-grained Text-to-Image Synthesis

RiFeGAN: Rich Feature Generation for Text-to-Image Synthesis From Prior Knowledge

AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks

RiFeGAN2: Rich Feature Generation for Text-to-Image Synthesis from Constrained Prior Knowledge

Object-driven Text-to-Image Synthesis via Adversarial Training

Joint Sketch-Attribute Learning for Fine-Grained Face Synthesis.

KT-GAN: Knowledge-Transfer Generative Adversarial Network for Text-to-Image Synthesis

Perceptual Pyramid Adversarial Networks for Text-to-Image Synthesis.

R-GAN: Exploring Human-like Way for Reasonable Text-to-Image Synthesis via Generative Adversarial Networks

TEXT TO IMAGE SYNTHESIS WITH ERUDITE GENERATIVE ADVERSARIAL NETWORKS

Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis.

DAE-GAN: Dynamic Aspect-aware GAN for Text-to-Image Synthesis

Attn-Eh ALN: complex text-to-image generation with attention-enhancing adversarial learning networks

Adma-GAN: Attribute-Driven Memory Augmented GANs for Text-to-Image Generation.

Attributes Aware Face Generation with Generative Adversarial Networks

Multi-Sentence Auxiliary Adversarial Networks for Fine-Grained Text-to-Image Synthesis

MGF-GAN: Multi Granularity Text Feature Fusion for Text-guided-Image Synthesis