Abstract:Generative adversarial networks (GANs) are challenging to train stably, and a promising remedy of injecting instance noise into the discriminator input has not been very effective in practice. In this paper, we propose Diffusion-GAN, a novel GAN framework that leverages a forward diffusion chain to generate Gaussian-mixture distributed instance noise. Diffusion-GAN consists of three components, including an adaptive diffusion process, a diffusion timestep-dependent discriminator, and a generator. Both the observed and generated data are diffused by the same adaptive diffusion process. At each diffusion timestep, there is a different noise-to-data ratio and the timestep-dependent discriminator learns to distinguish the diffused real data from the diffused generated data. The generator learns from the discriminator's feedback by backpropagating through the forward diffusion chain, whose length is adaptively adjusted to balance the noise and data levels. We theoretically show that the discriminator's timestep-dependent strategy gives consistent and helpful guidance to the generator, enabling it to match the true data distribution. We demonstrate the advantages of Diffusion-GAN over strong GAN baselines on various datasets, showing that it can produce more realistic images with higher stability and data efficiency than state-of-the-art GANs.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper proposes a new Generative Adversarial Network (GAN) framework—Diffusion-GAN, aiming to address some key issues in the traditional GAN training process, particularly training instability and the poor effect of instance noise injection. #### Main Issues: 1. **Training Stability**: Traditional GANs often encounter instability during training, including non-convergence, unstable training, and mode collapse. 2. **Instance Noise Injection**: Although injecting instance noise into the discriminator input is a potentially effective method, its practical application is not very effective. Finding an appropriate noise distribution is very difficult, and instance noise in high-dimensional data often performs poorly. #### Solutions: The paper proposes Diffusion-GAN to address the above issues through the following methods: - **Forward Diffusion Chain**: Utilizes the forward diffusion process to generate instance noise with a Gaussian mixture distribution. - **Time Step Dependent Discriminator**: Designs a discriminator that depends on the diffusion time step, capable of distinguishing real data from generated data at different time steps. - **Adaptive Diffusion Intensity**: Dynamically adjusts the length of the diffusion process to balance noise and data levels, thereby improving training stability and data efficiency. ### Experimental Results Extensive experiments validate the superior performance of Diffusion-GAN on multiple benchmark datasets, especially in terms of image generation quality and diversity. The experimental results show that Diffusion-GAN significantly outperforms existing advanced GAN models on most datasets and also improves performance under limited data conditions. Additionally, theoretical analysis proves the effectiveness of this method and demonstrates its generality across different tasks.

Diffusion-GAN: Training GANs with Diffusion

Tackling the Generative Learning Trilemma with Denoising Diffusion GANs

A New Distributed Method for Training Generative Adversarial Networks

Efficient Transfer Learning in Diffusion Models via Adversarial Noise

Distilling Diffusion Models into Conditional GANs

Differentiable Augmentation for Data-Efficient GAN Training

Exploring Guided Sampling of Conditional GANs

Generation diffusion degradation: Simple and efficient design for blind super-resolution

Latent Denoising Diffusion GAN: Faster sampling, Higher image quality

Accelerating Video Diffusion Models via Distribution Matching

SinDiffusion: Learning a Diffusion Model from a Single Natural Image

AdvDiff: Generating Unrestricted Adversarial Examples using Diffusion Models

Control3Diff: Learning Controllable 3D Diffusion Models from Single-view Images

DuDGAN: Improving Class-Conditional GANs via Dual-Diffusion

GUD: Generation with Unified Diffusion

Enhancing Unsupervised Speech Recognition with Diffusion GANs

Can Diffusion Model Conditionally Generate Astrophysical Images?

Towards the Gradient Vanishing, Divergence Mismatching and Mode Collapse of Generative Adversarial Nets

Diffusion Noise Feature: Accurate and Fast Generated Image Detection

Conditional GAN for Enhancing Diffusion Models in Efficient and Authentic Global Gesture Generation from Audios

Dist-GAN: An Improved GAN using Distance Constraints