Abstract:Synthesizing complex images from text presents challenging. Compared to autoregressive and diffusion model-based methods, Generative Adversarial Network-based methods have significant advantages in terms of computational cost and generation efficiency yet remain two limitations: first, these methods often refine all features output from the previous stage indiscriminately, without considering these features are initialized gradually during the generation process; second, the sparse semantic constraints provided by the text description are typically ineffective for refining fine-grained features. These issues complicate the balance between generation quality, computational cost and inference speed. To address these issues, we propose a Multi-granularity Feature Aware Enhancement GAN (MFAE-GAN), which allows the refinement process to match the order of different granularity features being initialized. Specifically, MFAE-GAN (1) samples category-related coarse-grained features and instance-level detail-related fine-grained features at different generation stages based on different attention mechanisms in Coarse-grained Feature Enhancement (CFE) and Fine-grained Feature Enhancement (FFE) to guide the generation process spatially, (2) provides denser semantic constraints than textual semantic information through Multi-granularity Features Adaptive Batch Normalization (MFA-BN) in the process of refining fine-grained features, and (3) adopts a Global Semantics Preservation (GSP) to avoid the loss of global semantics when sampling features continuously. Extensive experimental results demonstrate that our MFAE-GAN is competitive in terms of both image generation quality and efficiency.

Multi-Sentence Auxiliary Adversarial Networks for Fine-Grained Text-to-Image Synthesis

MF-GAN: Multi-conditional Fusion Generative Adversarial Network for Text-to-Image Synthesis

Multi-scale Dual-Modal Generative Adversarial Networks for Text-to-image Synthesis

Multi-Semantic Fusion Generative Adversarial Network for Text-to-Image Generation

Multi-Sentence Complementarily Generation for Text-to-Image Synthesis

SAM-GAN: Self-Attention supporting Multi-stage Generative Adversarial Networks for text-to-image synthesis

CD-GAN: Commonsense-Driven Generative Adversarial Network with Hierarchical Refinement for Text-to-Image Synthesis

DMF-GAN: Deep Multimodal Fusion Generative Adversarial Networks for Text-to-Image Synthesis

SAW-GAN: Multi-granularity Text Fusion Generative Adversarial Networks for text-to-image generation

Cross-Modal Semantic Matching Generative Adversarial Networks for Text-to-Image Synthesis

Self-Modulated Feature Fusion GAN for Text-to-Image Synthesis

Semantics-Enhanced Adversarial Nets for Text-to-Image Synthesis.

Exploring Sparse MoE in GANs for Text-conditioned Image Synthesis.

Word self-update contrastive adversarial networks for text-to-image synthesis

Text to Image Synthesis with Multi-Granularity Feature Aware Enhancement Generative Adversarial Networks

Text-to-Image Generation with Multiscale Semantic Context-Aware Generative Adversarial Networks

Dualattn-GAN: Text to Image Synthesis with Dual Attentional Generative Adversarial Network.

KT-GAN: Knowledge-Transfer Generative Adversarial Network for Text-to-Image Synthesis

SS-GANs: Text-to-Image via Stage by Stage Generative Adversarial Networks

Attention-Bridged Modal Interaction for Text-to-Image Generation

Fine-Grained Semantic Image Synthesis with Object-Attention Generative Adversarial Network