Abstract:<p>Synthesizing photo-realistic images based on text descriptions is a challenging task in the field of computer vision. Although generative adversarial networks have made significant breakthroughs in this task, they still face huge challenges in generating high-quality visually realistic images consistent with the semantics of text. Generally, existing text-to-image methods accomplish this task with two steps, that is, first generating an initial image with a rough outline and color, and then gradually yielding the image within high-resolution from the initial image. However, one drawback of these methods is that, if the quality of the initial image generation is not high, it is hard to generate a satisfactory high-resolution image. In this paper, we propose SAM-GAN, Self-Attention supporting Multi-stage Generative Adversarial Networks, for text-to-image synthesis. With the self-attention mechanism, the model can establish the multi-level dependence of the image and fuse the sentence- and word-level visual-semantic vectors, to improve the quality of the generated image. Furthermore, a multi-stage perceptual loss is introduced to enhance the semantic similarity between the synthesized image and the real image, thus enhancing the visual-semantic consistency between text and images. For the diversity of the generated images, a mode seeking regularization term is integrated into the model. The results of extensive experiments and ablation studies, which were conducted in the Caltech-UCSD Birds and Microsoft Common Objects in Context datasets, show that our model is superior to competitive models in text-to-image synthesis.</p>

Text-to-Image Synthesis via Visual-Memory Creative Adversarial Network.

Multi-Sentence Complementarily Generation for Text-to-Image Synthesis

DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesis

Object-driven Text-to-Image Synthesis via Adversarial Training

Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis.

Multi-Semantic Fusion Generative Adversarial Network for Text-to-Image Generation

Multimodal Fusion Generative Adversarial Network for Image Synthesis

Cross-Modal Semantic Matching Generative Adversarial Networks for Text-to-Image Synthesis

CPGAN: Content-Parsing Generative Adversarial Networks for Text-to-Image Synthesis.

Perceptual Pyramid Adversarial Networks for Text-to-Image Synthesis.

Adma-GAN: Attribute-Driven Memory Augmented GANs for Text-to-Image Generation.

Multi-Sentence Auxiliary Adversarial Networks for Fine-Grained Text-to-Image Synthesis

DMF-GAN: Deep Multimodal Fusion Generative Adversarial Networks for Text-to-Image Synthesis

CD-GAN: Commonsense-Driven Generative Adversarial Network with Hierarchical Refinement for Text-to-Image Synthesis

Photographic Text-to-Image Synthesis with a Hierarchically-nested Adversarial Network

Self-Modulated Feature Fusion GAN for Text-to-Image Synthesis

End-to-End Text-to-Image Synthesis with Spatial Constrains

MF-GAN: Multi-conditional Fusion Generative Adversarial Network for Text-to-Image Synthesis

Text-to-Image Generation with Multiscale Semantic Context-Aware Generative Adversarial Networks

SAM-GAN: Self-Attention supporting Multi-stage Generative Adversarial Networks for text-to-image synthesis

Adaptive Forgetting, Drafting and Comprehensive Guiding: Text-to-Image Synthesis with Hierarchical Generative Adversarial Networks