Abstract:Text-to-image generation (T2I) aims to produce visually compelling images while maintaining a high degree of semantic consistency with textual descriptions. Despite the impressive progress made by existing methods, there are problems with limited details of synthesized images and insufficient correlation between the provided text description and the generated images. To address these issues, we propose a Context-Aware Generative Adversarial Network (CA-GAN), which generates images aligned with the input text representations. Specifically, the Context-Aware Block (CA-Block) learns a semantic-adaptive transformation based on text style, enabling the effective fusion of text descriptions and image features for high-quality image generation with better language-vision matching. Furthermore, we propose an Attention Convolution Module (ACM) to identify greater representative traits and avoid the inability to capitalize on non-local contextual information, which enables our model to generate images that exhibit numerous detailed attributes while maintaining high-quality and semantic consistency. Thereafter, we integrate self-attention with convolution to enhance feature maps to reinforce the semantic information in the discriminator, emphasizing critical feature channels while suppressing extraneous information, ultimately yielding more detailed and richer images. The experimental results demonstrate the superiority of our method over the SOTA approaches. In addition, further studies confirm the effectiveness of the generated visual details, which exhibit a high degree of alignment with the input text descriptions. Notably, our attention mechanism showcases cooperative effects contributing to overall performance improvement. The code is available at: https://github.com/hylneu/CAGAN.

Dualattn-GAN: Text to Image Synthesis with Dual Attentional Generative Adversarial Network.

DGattGAN: Cooperative Up-Sampling Based Dual Generator Attentional GAN on Text-to-Image Synthesis

DTGAN: Dual Attention Generative Adversarial Networks for Text-to-Image Generation

DualG-GAN, a Dual-channel Generator based Generative Adversarial Network for text-to-face synthesis

Multi-scale Dual-Modal Generative Adversarial Networks for Text-to-image Synthesis

AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks

Dual Attention GANs for Semantic Image Synthesis

DE-GAN: Text-to-image Synthesis with Dual and Efficient Fusion Model

Hybrid Attention Driven Text-To-Image Synthesis Via Generative Adversarial Networks

GACnet-Text-to-Image Synthesis With Generative Models Using Attention Mechanisms With Contrastive Learning

Object-driven Text-to-Image Synthesis via Adversarial Training

Text to Image Synthesis with Bidirectional Generative Adversarial Network.

Cross-modal Feature Alignment Based Hybrid Attentional Generative Adversarial Networks for Text-to-image Synthesis

SAM-GAN: Self-Attention supporting Multi-stage Generative Adversarial Networks for text-to-image synthesis

MF-GAN: Multi-conditional Fusion Generative Adversarial Network for Text-to-Image Synthesis

Attention-Bridged Modal Interaction for Text-to-Image Generation

Class-Balanced Text to Image Synthesis with Attentive Generative Adversarial Network

Self-Modulated Feature Fusion GAN for Text-to-Image Synthesis

KT-GAN: Knowledge-Transfer Generative Adversarial Network for Text-to-Image Synthesis

Language-vision Matching for Text-to-image Synthesis with Context-Aware GAN