Abstract:Existing generative adversarial network (GAN) based conditional imagegenerative models typically produce fixed output for the same conditionalinput, which is unreasonable for highly subjective tasks, such as large-maskimage inpainting or style transfer. On the other hand, GAN-based diverse imagegenerative methods require retraining/fine-tuning the network or designingcomplex noise injection functions, which is computationally expensive,task-specific, or struggle to generate high-quality results. Given that manydeterministic conditional image generative models have been able to producehigh-quality yet fixed results, we raise an intriguing question: is it possiblefor pre-trained deterministic conditional image generative models to generatediverse results without changing network structures or parameters? To answerthis question, we re-examine the conditional image generation tasks from theperspective of adversarial attack and propose a simple and efficient plug-inprojected gradient descent (PGD) like method for diverse and controllable imagegeneration. The key idea is attacking the pre-trained deterministic generativemodels by adding a micro perturbation to the input condition. In this way,diverse results can be generated without any adjustment of network structuresor fine-tuning of the pre-trained models. In addition, we can also control thediverse results to be generated by specifying the attack direction according toa reference text or image. Our work opens the door to applying adversarialattack to low-level vision tasks, and experiments on various conditional imagegeneration tasks demonstrate the effectiveness and superiority of the proposedmethod.

Interactive Grayscale Image Colorization with Generative Adversarial Networks

Creative and Diverse Artwork Generation Using Adversarial Networks

Double-Channel Guided Generative Adversarial Network for Image Colorization

Attack Deterministic Conditional Image Generative Models for Diverse and Controllable Generation

Unsupervised Image-to-Image Translation with Generative Adversarial Networks.

Towards Vivid and Diverse Image Colorization with Generative Color Prior

UGSC‐GAN: User‐guided sketch colorization with deep convolution generative adversarial networks

Unsupervised Diverse Colorization via Generative Adversarial Networks

SCGAN: Saliency Map-guided Colorization with Generative Adversarial Network

Automatic Video Colorization using 3D Conditional Generative Adversarial Networks

A Novel Grayscale Image Steganography via Generative Adversarial Network

Grayscale Image Colorization with GAN and CycleGAN in Different Image Domain

Real-Time User-Guided Image Colorization with Learned Deep Priors

Image Colorization Algorithm Based on Deep Learning

Interactive Deep Colorization With Simultaneous Global and Local Inputs

SPDGAN: A Generative Adversarial Network based on SPD Manifold Learning for Automatic Image Colorization

GPU-Accelerated Near Real-Time Image Colorization

Interactive Deep Image Colorization of Quality

Satellite Remote Sensing Grayscale Image Colorization Based on Denoising Generative Adversarial Network

A learning-based approach for automatic image and video colorization

Colourization of Black and White Images using Deep Learning