Abstract:Synthesizing high-resolution realistic images from text descriptions is a challenging task. Almost all existing text-to-image methods employ stacked generative adversarial networks as the backbone, utilize cross-modal attention mechanisms to fuse text and image features, and use extra networks to ensure text-image semantic consistency. The existing text-to-image models have three problems: 1) For the backbone, there are multiple generators and discriminators stacked for generating different scales of images making the training process slow and inefficient. 2) For semantic consistency, the existing models employ extra networks to ensure the semantic consistency increasing the training complexity and bringing an additional computational cost. 3) For the text-image feature fusion method, cross-modal attention is only applied a few times during the generation process due to its computational cost impeding fusing the text and image features deeply. To solve these limitations, we propose 1) a novel simplified text-to-image backbone which is able to synthesize high-quality images directly by one pair of generator and discriminator, 2) a novel regularization method called Matching-Aware zero-centered Gradient Penalty which promotes the generator to synthesize more realistic and text-image semantic consistent images without introducing extra networks, 3) a novel fusion module called Deep Text-Image Fusion Block which can exploit the semantics of text descriptions effectively and fuse text and image features deeply during the generation process. Compared with the previous text-to-image models, our DF-GAN is simpler and more efficient and achieves better performance. Extensive experiments and ablation studies on both Caltech-UCSD Birds 200 and COCO datasets demonstrate the superiority of the proposed model in comparison to state-of-the-art models.

RiFeGAN2: Rich Feature Generation for Text-to-Image Synthesis from Constrained Prior Knowledge

RiFeGAN: Rich Feature Generation for Text-to-Image Synthesis From Prior Knowledge

Learn, Imagine and Create: Text-to-Image Generation from Prior Knowledge.

Diversified text-to-image generation via deep mutual information estimation

Specific Diverse Text-to-Image Synthesis Via Exemplar Guidance

ISF-GAN: Imagine, Select, and Fuse with GPT-Based Text Enrichment for Text-to-Image Synthesis

SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks.

Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis.

Text-to-image Generation Based on Spatial-Channel Attention and Semantic Redescription

Tf-Gan: Text Feature Fusion Gan for Text-to-Image Generation

Text2FaceGAN: Face Generation from Fine Grained Textual Descriptions

Enriching Attributes from Knowledge Graph for Fine-grained Text-to-Image Synthesis.

DF-GAN: Deep Fusion Generative Adversarial Networks for Text-to-Image Synthesis.

R-GAN: Exploring Human-like Way for Reasonable Text-to-Image Synthesis via Generative Adversarial Networks

DMF-GAN: Deep Multimodal Fusion Generative Adversarial Networks for Text-to-Image Synthesis

KT-GAN: Knowledge-Transfer Generative Adversarial Network for Text-to-Image Synthesis

SAW-GAN: Multi-granularity Text Fusion Generative Adversarial Networks for text-to-image generation

RII-GAN: Multi-scaled Aligning-Based Reversed Image Interaction Network for Text-to-Image Synthesis

SAM-GAN: Self-Attention supporting Multi-stage Generative Adversarial Networks for text-to-image synthesis

Expressive Text-to-Image Generation with Rich Text

DAE-GAN: Dynamic Aspect-aware GAN for Text-to-Image Synthesis