GenCo: Generative Co-training for Generative Adversarial Networks with Limited Data

Kaiwen Cui,Jiaxing Huang,Zhipeng Luo,Gongjie Zhang,Fangneng Zhan,Shijian Lu

DOI: https://doi.org/10.48550/arXiv.2110.01254

2021-12-06

Abstract:Training effective Generative Adversarial Networks (GANs) requires large amounts of training data, without which the trained models are usually sub-optimal with discriminator over-fitting. Several prior studies address this issue by expanding the distribution of the limited training data via massive and hand-crafted data augmentation. We handle data-limited image generation from a very different perspective. Specifically, we design GenCo, a Generative Co-training network that mitigates the discriminator over-fitting issue by introducing multiple complementary discriminators that provide diverse supervision from multiple distinctive views in training. We instantiate the idea of GenCo in two ways. The first way is Weight-Discrepancy Co-training (WeCo) which co-trains multiple distinctive discriminators by diversifying their parameters. The second way is Data-Discrepancy Co-training (DaCo) which achieves co-training by feeding discriminators with different views of the input images (e.g., different frequency components of the input images). Extensive experiments over multiple benchmarks show that GenCo achieves superior generation with limited training data. In addition, GenCo also complements the augmentation approach with consistent and clear performance gains when combined.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the challenge of generating high - quality images with limited training data, especially the discriminator over - fitting problem in Generative Adversarial Networks (GANs). When the number of available training samples is small, traditional GAN models are prone to discriminator over - fitting, resulting in a decline in the quality of generated images. To solve this problem, the paper proposes a new framework - GenCo (Generative Co - training). By introducing multiple complementary discriminators to learn limited data from different perspectives, the discriminator over - fitting can be alleviated and the image generation quality under limited data conditions can be improved.

Specifically, GenCo has designed two instantiated co - training methods:
1. **Weight - Discrepancy Co - training (WeCo)**: By introducing the weight - discrepancy loss to train multiple discriminators with different parameters, these discriminators can provide supervision signals from different angles.
2. **Data - Discrepancy Co - training (DaCo)**: By inputting different views of the image (for example, different frequency components) to different discriminators to achieve co - training, so that the discriminators can capture more details of the input data.

Through these two methods, GenCo can effectively alleviate the discriminator over - fitting on a limited data set and improve the quality of generated images. Experimental results show that GenCo not only performs excellently in multiple benchmark tests, but also can further improve performance when combined with existing data augmentation methods.