Abstract:Existing generative adversarial network (GAN) based conditional imagegenerative models typically produce fixed output for the same conditionalinput, which is unreasonable for highly subjective tasks, such as large-maskimage inpainting or style transfer. On the other hand, GAN-based diverse imagegenerative methods require retraining/fine-tuning the network or designingcomplex noise injection functions, which is computationally expensive,task-specific, or struggle to generate high-quality results. Given that manydeterministic conditional image generative models have been able to producehigh-quality yet fixed results, we raise an intriguing question: is it possiblefor pre-trained deterministic conditional image generative models to generatediverse results without changing network structures or parameters? To answerthis question, we re-examine the conditional image generation tasks from theperspective of adversarial attack and propose a simple and efficient plug-inprojected gradient descent (PGD) like method for diverse and controllable imagegeneration. The key idea is attacking the pre-trained deterministic generativemodels by adding a micro perturbation to the input condition. In this way,diverse results can be generated without any adjustment of network structuresor fine-tuning of the pre-trained models. In addition, we can also control thediverse results to be generated by specifying the attack direction according toa reference text or image. Our work opens the door to applying adversarialattack to low-level vision tasks, and experiments on various conditional imagegeneration tasks demonstrate the effectiveness and superiority of the proposedmethod.

Conditional Image Generation Using Feature-Matching Gan

Attack Deterministic Conditional Image Generative Models for Diverse and Controllable Generation

Statistics Enhancement Generative Adversarial Networks for Diverse Conditional Image Synthesis

Synthesizing Contrast-enhanced Computed Tomography Images with an Improved Conditional Generative Adversarial Network

Conditional Generative Adversarial Nets

Ways of Conditioning Generative Adversarial Networks

A one-to-many conditional generative adversarial network framework for multiple image-to-image translations

Guiding GANs: How to control non-conditional pre-trained GANs for conditional image generation

Incremental Focal Loss GANs.

Fast-converging Conditional Generative Adversarial Networks for Image Synthesis

Variational Conditional GAN for Fine-grained Controllable Image Generation

Semantic Image Analogy with a Conditional Single-Image GAN

Matchinggan: Matching-Based Few-Shot Image Generation

Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis

Conditional Image Synthesis With Auxiliary Classifier GANs

Leveraging Conditional Generative Adversarial Networks (cGANs) for enhanced artistic creation: Exploring quality improvement and content control through conditional inputs

Structured Generative Adversarial Networks.

Feature-aware conditional GAN for category text generation

Diverse Image Generation via Self-Conditioned GANs

DivCo: Diverse Conditional Image Synthesis via Contrastive Generative Adversarial Network

Antimode collapse generative adversarial networks