Abstract:Generative Adversarial Networks (GANs) have recently achieved significant improvement on paired/unpaired image-to-image translation, such as photo$\rightarrow$ sketch and artist painting style transfer. However, existing models can only be capable of transferring the low-level information (e.g. color or texture changes), but fail to edit high-level semantic meanings (e.g., geometric structure or content) of objects. On the other hand, while some researches can synthesize compelling real-world images given a class label or caption, they cannot condition on arbitrary shapes or structures, which largely limits their application scenarios and interpretive capability of model results. In this work, we focus on a more challenging semantic manipulation task, which aims to modify the semantic meaning of an object while preserving its own characteristics (e.g. viewpoints and shapes), such as cow$\rightarrow$sheep, motor$\rightarrow$ bicycle, cat$\rightarrow$dog. To tackle such large semantic changes, we introduce a contrasting GAN (contrast-GAN) with a novel adversarial contrasting objective. Instead of directly making the synthesized samples close to target data as previous GANs did, our adversarial contrasting objective optimizes over the distance comparisons between samples, that is, enforcing the manipulated data be semantically closer to the real data with target category than the input data. Equipped with the new contrasting objective, a novel mask-conditional contrast-GAN architecture is proposed to enable disentangle image background with object semantic changes. Experiments on several semantic manipulation tasks on ImageNet and MSCOCO dataset show considerable performance gain by our contrast-GAN over other conditional GANs. Quantitative results further demonstrate the superiority of our model on generating manipulated results with high visual fidelity and reasonable object semantics.

Edge Guided GANs with Multi-Scale Contrastive Learning for Semantic Image Synthesis

Edge Guided GANs with Contrastive Learning for Semantic Image Synthesis

Fine-grained Semantic Constraint in Image Synthesis

Dual Attention GANs for Semantic Image Synthesis

SIMGAN: Photo-Realistic Semantic Image Manipulation Using Generative Adversarial Networks.

Synthesizing Contrast-enhanced Computed Tomography Images with an Improved Conditional Generative Adversarial Network

OA-GAN: Organ-Aware Generative Adversarial Network for Synthesizing Contrast-Enhanced Medical Images

GAN for Semantic Image Synthesis With Laplacian Pyramid and Multi-Scale Channel Attention

Fine-Grained Semantic Image Synthesis with Object-Attention Generative Adversarial Network

Local and Global GANs with Semantic-Aware Upsampling for Image Generation

Generative Semantic Manipulation with Contrasting GAN

Remote Sensing Image Synthesis Via Semantic Embedding Generative Adversarial Networks.

Global-Affine and Local-Specific Generative Adversarial Network for Semantic-Guided Image Generation.

Semantic Image Analogy with a Conditional Single-Image GAN

Ea-GANs: Edge-aware Generative Adversarial Networks for Cross-modality MR Image Synthesis.

Learning to Predict Layout-to-image Conditional Convolutions for Semantic Image Synthesis.

Unlocking Pre-trained Image Backbones for Semantic Image Synthesis

Semantics-Enhanced Adversarial Nets for Text-to-Image Synthesis.

Multi-Channel Attention Selection GAN With Cascaded Semantic Guidance for Cross-View Image Translation

Cross-Modal Semantic Matching Generative Adversarial Networks for Text-to-Image Synthesis

Multi-level Gate Feature Aggregation with Spatially Adaptive Batch-Instance Normalization for Semantic Image Synthesis