Abstract:Text-to-image generation aims at generating realistic images which are semantically consistent with the given text. Previous works mainly adopt the multi-stage architecture by stacking generator-discriminator pairs to engage multiple adversarial training, where the text semantics used to provide generation guidance remain static across all stages. This work argues that text features at each stage should be adaptively re-composed conditioned on the status of the historical stage (\emphi.e., historical stage's text and image features) to provide diversified and accurate semantic guidance during the coarse-to-fine generation process. We thereby propose a novel Dynamical Semantic Evolution GAN (DSE-GAN) to re-compose each stage's text features under a novel single adversarial multi-stage architecture. Specifically, we design (1) Dynamic Semantic Evolution (DSE) module, which first aggregates historical image features to summarize the generative feedback, and then dynamically selects words required to be re-composed at each stage as well as re-composed them by dynamically enhancing or suppressing different granularity subspace's semantics. (2) Single Adversarial Multi-stage Architecture (SAMA), which extends the previous structure by eliminating complicated multiple adversarial training requirements and therefore allows more stages of text-image interactions, and finally facilitates the DSE module. We conduct comprehensive experiments and show that DSE-GAN achieves 7.48% and 37.8% relative FID improvement on two widely used benchmarks, i.e., CUB-200 and MSCOCO, respectively.

Penalty-based Sequence Generative Adversarial Networks with Enhanced Transformer for Text Generation

Adversarial Sub-sequence for Text Generation.

Adversarial Subsequences for Unconditional Text Generation

Sequence Generative Adversarial Nets with a Conditional Discriminator

Text Generation Based on Generative Adversarial Nets with Latent Variable

Adversarial Text Generation Via Sequence Contrast Discrimination.

SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient

A Sequence-to-Sequence&Set Model for Text-to-Table Generation

A Text GAN for Language Generation with Non-Autoregressive Generator

Diversity-Promoting GAN: A Cross-Entropy Based Generative Adversarial Network for Diversified Text Generation

SentiGAN: Generating Sentimental Texts Via Mixture Adversarial Networks.

ReGAN: RE[LAX|BAR|INFORCE] based Sequence Generation using GANs

Improving Conditional Sequence Generative Adversarial Networks by Stepwise Evaluation

DSE-GAN: Dynamic Semantic Evolution Generative Adversarial Network for Text-to-Image Generation

Long Text Generation via Adversarial Training with Leaked Information

Text Feature Adversarial Learning for Text Generation With Knowledge Transfer From GPT2

Adversarial Feature Matching for Text Generation

Explicit Syntactic Guidance for Neural Text Generation

Unlocking the Power of GANs in Non-Autoregressive Text Generation

DTGAN: Dual Attention Generative Adversarial Networks for Text-to-Image Generation