Stabilizing GANs' Training with Brownian Motion Controller

Tianjiao Luo,Ziyu Zhu,Jianfei Chen,Jun Zhu

2023-06-18

Abstract:The training process of generative adversarial networks (GANs) is unstable and does not converge globally. In this paper, we examine the stability of GANs from the perspective of control theory and propose a universal higher-order noise-based controller called Brownian Motion Controller (BMC). Starting with the prototypical case of Dirac-GANs, we design a BMC to retrieve precisely the same but reachable optimal equilibrium. We theoretically prove that the training process of DiracGANs-BMC is globally exponential stable and derive bounds on the rate of convergence. Then we extend our BMC to normal GANs and provide implementation instructions on GANs-BMC. Our experiments show that our GANs-BMC effectively stabilizes GANs' training under StyleGANv2-ada frameworks with a faster rate of convergence, a smaller range of oscillation, and better performance in terms of FID score.

Machine Learning,Dynamical Systems

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to address the issue of instability and lack of global convergence in the training process of Generative Adversarial Networks (GANs). Specifically, the paper proposes a method based on the Brownian Motion Controller (BMC) to stabilize the training process of GANs. By injecting noise into the training dynamics, BMC ensures global exponential stability of the training process and improves the convergence speed. ### Main Contributions: 1. **Theoretical Proof**: For a simplified version of Dirac-GANs, the paper proves that BMC can make the training process globally exponentially stable. 2. **Experimental Validation**: Experiments demonstrate that BMC is not only theoretically effective but also significantly improves the training stability of various GAN architectures (such as StyleGANv2-ada) in practical applications, reduces oscillation range, and enhances image quality (measured by FID score). ### Summary: By introducing BMC, the paper addresses the instability issue in GANs training, achieving faster and more stable convergence, and demonstrates its superiority across multiple datasets.

Stabilizing GANs' Training with Brownian Motion Controller

BrGANs: Stabilizing GANs' Training Process with Brownian Motion Control

Understanding and Stabilizing GANs' Training Dynamics with Control Theory

Smoothness and Stability in GANs

Stabilizing Training of Generative Adversarial Nets via Langevin Stein Variational Gradient Descent

Direct Adversarial Training: A New Approach for Stabilizing The Training Process of GANs

A New Perspective on Stabilizing GANs Training: Direct Adversarial Training

Wasserstein Bounded Generative Adversarial Networks

Kernel-Guided Training of Implicit Generative Models with Stability Guarantees

Convergence of GANs Training: A Game and Stochastic Control Methodology

Stabilizing Training of Generative Adversarial Networks through Regularization

An Automatic Control Perspective on Parameterizing Generative Adversarial Network

Lipschitz Constrained GANs via Boundedness and Continuity

Stabilizing Generative Adversarial Networks: A Survey

On Convergence and Stability of GANs

Instability analysis for generative adversarial networks and its solving techniques

Towards a Better Understanding and Regularization of GAN Training Dynamics

Addressing GAN Training Instabilities via Tunable Classification Losses

Training Generative Adversarial Networks by Solving Ordinary Differential Equations

On Stabilizing Generative Adversarial Training with Noise