Improving GANs with A Dynamic Discriminator

Ceyuan Yang,Yujun Shen,Yinghao Xu,Deli Zhao,Bo Dai,Bolei Zhou

DOI: https://doi.org/10.48550/arXiv.2209.09897

2022-09-21

Abstract:Discriminator plays a vital role in training generative adversarial networks (GANs) via distinguishing real and synthesized samples. While the real data distribution remains the same, the synthesis distribution keeps varying because of the evolving generator, and thus effects a corresponding change to the bi-classification task for the discriminator. We argue that a discriminator with an on-the-fly adjustment on its capacity can better accommodate such a time-varying task. A comprehensive empirical study confirms that the proposed training strategy, termed as DynamicD, improves the synthesis performance without incurring any additional computation cost or training objectives. Two capacity adjusting schemes are developed for training GANs under different data regimes: i) given a sufficient amount of training data, the discriminator benefits from a progressively increased learning capacity, and ii) when the training data is limited, gradually decreasing the layer width mitigates the over-fitting issue of the discriminator. Experiments on both 2D and 3D-aware image synthesis tasks conducted on a range of datasets substantiate the generalizability of our DynamicD as well as its substantial improvement over the baselines. Furthermore, DynamicD is synergistic to other discriminator-improving approaches (including data augmentation, regularizers, and pre-training), and brings continuous performance gain when combined for learning GANs.

Computer Vision and Pattern Recognition,Image and Video Processing

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how the discriminator can better adapt to the constantly changing synthetic distribution of the generator during the training process of Generative Adversarial Networks (GANs). Specifically, the paper points out that in GANs, as the generator's ability improves, the distribution of synthetic data is also constantly changing, which makes the binary classification task faced by the discriminator (distinguishing real samples from synthetic samples) also change dynamically. Traditional discriminators with fixed capacity may not be able to cope well with this temporal change, thus affecting the performance of the generator. Therefore, the author proposes a method of dynamically adjusting the discriminator capacity (DynamicD) to better adapt to the task requirements of this dynamic change, thereby improving the quality of image synthesis.

The paper verifies through experiments that under different data scales (sufficient data and limited data), dynamically adjusting the discriminator capacity can significantly improve the synthetic performance of GANs without increasing additional computational costs or training objectives. In addition, the paper also explores the synergy between DynamicD and other discriminator improvement methods (such as data augmentation, regularization, pre - training, etc.), indicating that the combined use of these methods can further enhance the learning effect of GANs.