Abstract:Synthesizing high-resolution realistic images from text descriptions is a challenging task. Almost all existing text-to-image methods employ stacked generative adversarial networks as the backbone, utilize cross-modal attention mechanisms to fuse text and image features, and use extra networks to ensure text-image semantic consistency. The existing text-to-image models have three problems: 1) For the backbone, there are multiple generators and discriminators stacked for generating different scales of images making the training process slow and inefficient. 2) For semantic consistency, the existing models employ extra networks to ensure the semantic consistency increasing the training complexity and bringing an additional computational cost. 3) For the text-image feature fusion method, cross-modal attention is only applied a few times during the generation process due to its computational cost impeding fusing the text and image features deeply. To solve these limitations, we propose 1) a novel simplified text-to-image backbone which is able to synthesize high-quality images directly by one pair of generator and discriminator, 2) a novel regularization method called Matching-Aware zero-centered Gradient Penalty which promotes the generator to synthesize more realistic and text-image semantic consistent images without introducing extra networks, 3) a novel fusion module called Deep Text-Image Fusion Block which can exploit the semantics of text descriptions effectively and fuse text and image features deeply during the generation process. Compared with the previous text-to-image models, our DF-GAN is simpler and more efficient and achieves better performance. Extensive experiments and ablation studies on both Caltech-UCSD Birds 200 and COCO datasets demonstrate the superiority of the proposed model in comparison to state-of-the-art models.

Super-Resolution of Text Image Based on Conditional Generative Adversarial Network

Text Image Super-Resolution by Image Matting and Text Label Supervision

Improving Text Image Resolution Using a Deep Generative Adversarial Network for Optical Character Recognition

Cartoon single-image super-resolution approach based on generative adversarial network

Learning to Super-Resolve Blurry Face and Text Images.

B2GAN: Bidirectional-branch generative adversarial network for text image super-resolution with structure preservation

CT-GAN: A conditional Generative Adversarial Network of transformer architecture for text-to-image

DTGAN: Dual Attention Generative Adversarial Networks for Text-to-Image Generation

Gram-GAN: Image Super-Resolution Based on Gram Matrix and Discriminator Perceptual Loss

Learning to Draw Text in Natural Images with Conditional Adversarial Networks

G-GANISR: Gradual generative adversarial network for image super resolution

A Super-Resolution Generative Adversarial Network with Simplified Gradient Penalty and Relativistic Discriminator

Unsupervised Image Super-Resolution Using Cycle-in-Cycle Generative Adversarial Networks

Csrgan: medical image super-resolution using a generative adversarial network

DF-GAN: Deep Fusion Generative Adversarial Networks for Text-to-Image Synthesis.

Adaptive Forgetting, Drafting and Comprehensive Guiding: Text-to-Image Synthesis with Hierarchical Generative Adversarial Networks

CF-GAN: cross-domain feature fusion generative adversarial network for text-to-image synthesis

Optimizing Generative Adversarial Networks for Image Super Resolution via Latent Space Regularization

SAW-GAN: Multi-granularity Text Fusion Generative Adversarial Networks for text-to-image generation

Real-World Image Super Resolution via Unsupervised Bi-directional Cycle Domain Transfer Learning based Generative Adversarial Network

Text Image Super-Resolution Guided by Text Structure and Embedding Priors