Abstract:This paper does not describe a novel method. Instead, it studies an incremental, yet must-know baseline given the recent progress in sparse neural network training and Generative Adversarial Networks (GANs). GANs have received an upsurging interest since being proposed due to the high quality of the generated data. While achieving increasingly impressive results, the resource demands associated with the large model size hinders the usage of GANs in resource-limited scenarios. For inference, the existing model compression techniques can reduce the model complexity with comparable performance. However, the training efficiency of GANs has less been explored due to the fragile training process of GANs. In this paper, we, for the first time, explore the possibility of directly training sparse GAN from scratch without involving any dense or pre-training steps. Even more unconventionally, our proposed method enables directly training sparse unbalanced GANs with an extremely sparse generator from scratch. Instead of training full GANs, we start with sparse GANs and dynamically explore the parameter space spanned over the generator throughout training. Such a sparse-to-sparse training procedure enhances the capacity of the highly sparse generator progressively while sticking to a fixed small parameter budget with appealing training and inference efficiency gains. Extensive experiments with modern GAN architectures validate the effectiveness of our method. Our sparsified GANs, trained from scratch in one single run, are able to outperform the ones learned by expensive iterative pruning and re-training. Perhaps most importantly, we find instead of inheriting parameters from expensive pre-trained GANs, directly training sparse GANs from scratch can be a much more efficient solution. For example, only training with a 80% sparse generator and a 70% sparse discriminator, our method can achieve even better performance than the dense BigGAN.

A Sparse Substitute for Deconvolution Layers in GANs.

Self Sparse Generative Adversarial Networks

MND-GAN: A Research on Image Deblurring Algorithm Based on Generative Adversarial Network

Don't Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance

UGC: Unified GAN Compression for Efficient Image-to-Image Translation

Joint Generative Image Deblurring Aided by Edge Attention Prior and Dynamic Kernel Selection

Incremental Focal Loss GANs.

Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models

SparseGAN: Sparse Generative Adversarial Network for Text Generation

A lightweight ensemble discriminator for Generative Adversarial Networks

Learning Efficient GANs Using Differentiable Masks and Co-Attention Distillation

Learning Efficient GANs using Differentiable Masks and co-Attention Distillation.

SPGAN: Siamese projection Generative Adversarial Networks

SP-GAN: Self-Growing and Pruning Generative Adversarial Networks

DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better

Sketch-then-Edit Generative Adversarial Network

Learning Sparse Latent Representations for Generator Model

Bi-Skip: A Motion Deblurring Network Using Self-paced Learning

A Super-Resolution Generative Adversarial Network with Simplified Gradient Penalty and Relativistic Discriminator

Flocking Birds of a Feather Together: Dual-step GAN Distillation Via Realer-Fake Samples

A Novel Generator With Auxiliary Branch for Improving GAN Performance