Abstract:In this paper, we focus on the semantic image synthesis task that aims at transferring semantic label maps to photo-realistic images. Existing methods lack effective semantic constraints to preserve the semantic information and ignore the structural correlations in both spatial and channel dimensions, leading to unsatisfactory blurry and artifact-prone results. To address these limitations, we propose a novel Dual Attention GAN (DAGAN) to synthesize photo-realistic and semantically-consistent images with fine details from the input layouts without imposing extra training overhead or modifying the network architectures of existing methods. We also propose two novel modules, i.e., position-wise Spatial Attention Module (SAM) and scale-wise Channel Attention Module (CAM), to capture semantic structure attention in spatial and channel dimensions, respectively. Specifically, SAM selectively correlates the pixels at each position by a spatial attention map, leading to pixels with the same semantic label being related to each other regardless of their spatial distances. Meanwhile, CAM selectively emphasizes the scalewise features at each channel by a channel attention map, which integrates associated features among all channel maps regardless of their scales. We finally sum the outputs of SAM and CAM to further improve feature representation. Extensive experiments on four challenging datasets show that DAGAN achieves remarkably better results than state-of-the-art methods, while using fewer model parameters. The source code and trained models are available at https://github.com/Ha0Tang/DAGAN.

Generative Adversarial Networks with Bi-directional Normalization for Semantic Image Synthesis

SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks.

Multi-level Gate Feature Aggregation with Spatially Adaptive Batch-Instance Normalization for Semantic Image Synthesis

Learning Semantic-aware Normalization for Generative Adversarial Networks.

Semantic Image Synthesis with Spatially-Adaptive Normalization

Dual Attention GANs for Semantic Image Synthesis

Text to Image Synthesis with Bidirectional Generative Adversarial Network.

Label-Guided Generative Adversarial Network for Realistic Image Synthesis

Asymmetric Generative Adversarial Networks with a New Attention Mechanism

CD-GAN: Commonsense-Driven Generative Adversarial Network with Hierarchical Refinement for Text-to-Image Synthesis

Progressive Semantic Image Synthesis via Generative Adversarial Network

Image enhancement with bi-directional normalization and color attention-guided generative adversarial networks

Remote Sensing Image Synthesis Via Semantic Embedding Generative Adversarial Networks.

Semantic Image Synthesis Via Diffusion Models

Background and Foreground Disentangled Generative Adversarial Network for Scene Image Synthesis

Photographic Text-to-Image Synthesis with a Hierarchically-nested Adversarial Network

Global-Affine and Local-Specific Generative Adversarial Network for Semantic-Guided Image Generation.

Retrieval-based Spatially Adaptive Normalization for Semantic Image Synthesis

Semantics-Enhanced Adversarial Nets for Text-to-Image Synthesis.

Semantic Invariant Cross-Domain Image Generation with Generative Adversarial Networks

Instance Map Based Image Synthesis with a Denoising Generative Adversarial Network.