Flooding Regularization for Stable Training of Generative Adversarial Networks

Iu Yahiro,Takashi Ishida,Naoto Yokoya

2024-03-18

Abstract:Generative Adversarial Networks (GANs) have shown remarkable performance in image generation. However, GAN training suffers from the problem of instability. One of the main approaches to address this problem is to modify the loss function, often using regularization terms in addition to changing the type of adversarial losses. This paper focuses on directly regularizing the adversarial loss function. We propose a method that applies flooding, an overfitting suppression method in supervised learning, to GANs to directly prevent the discriminator's loss from becoming excessively low. Flooding requires tuning the flood level, but when applied to GANs, we propose that the appropriate range of flood level settings is determined by the adversarial loss function, supported by theoretical analysis of GANs using the binary cross entropy loss. We experimentally verify that flooding stabilizes GAN training and can be combined with other stabilization techniques. We also show that by restricting the discriminator's loss to be no less than the flood level, the training proceeds stably even when the flood level is somewhat high.

Machine Learning,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the instability in the training of Generative Adversarial Networks (GANs). Specifically, GANs perform excellently in image - generation tasks, but their training process is prone to instability. One of the main reasons is that the loss of the discriminator becomes too low, which causes the discriminator to over - fit the current distribution, thus affecting the learning effect of the generator. This over - fitting will lead to the vanishing or exploding of gradients, and further make it difficult for the training to converge.

To solve this problem, the paper proposes a new regularization technique - "Flooding", which directly adjusts the loss of the discriminator to prevent its loss value from becoming too low, thereby stabilizing the training process of GANs. Based on the flooding method used to prevent over - fitting in supervised learning, the paper explores the possibility of applying it to GANs, and verifies the effectiveness of this method through theoretical analysis and experiments.