Abstract:Existing generative adversarial network (GAN) based conditional imagegenerative models typically produce fixed output for the same conditionalinput, which is unreasonable for highly subjective tasks, such as large-maskimage inpainting or style transfer. On the other hand, GAN-based diverse imagegenerative methods require retraining/fine-tuning the network or designingcomplex noise injection functions, which is computationally expensive,task-specific, or struggle to generate high-quality results. Given that manydeterministic conditional image generative models have been able to producehigh-quality yet fixed results, we raise an intriguing question: is it possiblefor pre-trained deterministic conditional image generative models to generatediverse results without changing network structures or parameters? To answerthis question, we re-examine the conditional image generation tasks from theperspective of adversarial attack and propose a simple and efficient plug-inprojected gradient descent (PGD) like method for diverse and controllable imagegeneration. The key idea is attacking the pre-trained deterministic generativemodels by adding a micro perturbation to the input condition. In this way,diverse results can be generated without any adjustment of network structuresor fine-tuning of the pre-trained models. In addition, we can also control thediverse results to be generated by specifying the attack direction according toa reference text or image. Our work opens the door to applying adversarialattack to low-level vision tasks, and experiments on various conditional imagegeneration tasks demonstrate the effectiveness and superiority of the proposedmethod.

Generative Adversarial Networks with Mixture of T-Distributions Noise for Diverse Image Generation.

Student's T-Generative Adversarial Networks

Attack Deterministic Conditional Image Generative Models for Diverse and Controllable Generation

Diversified text-to-image generation via deep mutual information estimation

Statistics Enhancement Generative Adversarial Networks for Diverse Conditional Image Synthesis

Creative and Diverse Artwork Generation Using Adversarial Networks

Specific Diverse Text-to-Image Synthesis Via Exemplar Guidance

UCTGAN: Diverse Image Inpainting Based on Unsupervised Cross-Space Translation

Multi-distribution Mixture Generative Adversarial Networks for Fitting Diverse Data Sets

Dual Discriminator Weighted Mixture Generative Adversarial Network for image generation

Feature-Grounded Single-Stage Text-to-Image Generation

Generate Desired Images from Trained Generative Adversarial Networks.

Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis

Diverse Image Generation via Self-Conditioned GANs

MIXGAN: Learning Concepts from Different Domains for Mixture Generation

Diverse Image Colorizaiton Via Deep Color Prior and Triplet Latent Regularization

Improved Triple Generative Adversarial Nets

Diversity-Promoting GAN: A Cross-Entropy Based Generative Adversarial Network for Diversified Text Generation

Teacher-Supervised Generative Adversarial Networks.

Hierarchical Mixtures of Generators for Adversarial Learning

Dual-Clustered Conditioning Toward GAN-Based Diverse Image Generation.