Abstract:Synthesizing high-resolution realistic images from text descriptions is a challenging task. Almost all existing text-to-image methods employ stacked generative adversarial networks as the backbone, utilize cross-modal attention mechanisms to fuse text and image features, and use extra networks to ensure text-image semantic consistency. The existing text-to-image models have three problems: 1) For the backbone, there are multiple generators and discriminators stacked for generating different scales of images making the training process slow and inefficient. 2) For semantic consistency, the existing models employ extra networks to ensure the semantic consistency increasing the training complexity and bringing an additional computational cost. 3) For the text-image feature fusion method, cross-modal attention is only applied a few times during the generation process due to its computational cost impeding fusing the text and image features deeply. To solve these limitations, we propose 1) a novel simplified text-to-image backbone which is able to synthesize high-quality images directly by one pair of generator and discriminator, 2) a novel regularization method called Matching-Aware zero-centered Gradient Penalty which promotes the generator to synthesize more realistic and text-image semantic consistent images without introducing extra networks, 3) a novel fusion module called Deep Text-Image Fusion Block which can exploit the semantics of text descriptions effectively and fuse text and image features deeply during the generation process. Compared with the previous text-to-image models, our DF-GAN is simpler and more efficient and achieves better performance. Extensive experiments and ablation studies on both Caltech-UCSD Birds 200 and COCO datasets demonstrate the superiority of the proposed model in comparison to state-of-the-art models.

Background and Foreground Disentangled Generative Adversarial Network for Scene Image Synthesis

FBC-GAN: Diverse and Flexible Image Synthesis via Foreground-Background Composition

Model-Guided Generative Adversarial Networks for Unsupervised Fine-Grained Image Generation

CD-GAN: Commonsense-Driven Generative Adversarial Network with Hierarchical Refinement for Text-to-Image Synthesis

Global-Affine and Local-Specific Generative Adversarial Network for Semantic-Guided Image Generation.

Exploiting Relationship for Complex-scene Image Generation

SSGAN: Image Generation from Freehand Scene Sketches

Scene Graph Disentanglement and Composition for Generalizable Complex Image Generation

Hierarchy Composition GAN for High-fidelity Image Synthesis

Attentive generative adversarial network to bridge multi-domain gap for image synthesis

Generative Adversarial Networks with Bi-directional Normalization for Semantic Image Synthesis

DF-GAN: Deep Fusion Generative Adversarial Networks for Text-to-Image Synthesis.

SCGAN: Semi-Centralized Generative Adversarial Network for Image Generation in Distributed Scenes

Remote Sensing Image Synthesis Via Semantic Embedding Generative Adversarial Networks.

Spatial Fusion GAN for Image Synthesis

Joint Generative Modeling of Scene Graphs and Images via Diffusion Models

Object-driven Text-to-Image Synthesis via Adversarial Training

ComGAN: Unsupervised Disentanglement and Segmentation via Image Composition

Compositional GAN: Learning Image-Conditional Binary Composition

Multimodal Fusion Generative Adversarial Network for Image Synthesis

Fine-Grained Semantic Image Synthesis with Object-Attention Generative Adversarial Network