Generative Adversarial Networks with Learnable Auxiliary Module for Image Synthesis

Yan Gan,Chenxue Yang,Mao Ye,Renjie Huang,Deqiang Ouyang

DOI: https://doi.org/10.1145/3653021

2024-03-17

Abstract:Training generative adversarial networks (GANs) for noise-to-image synthesis is a challenge task, primarily due to the instability of GANs’ training process. One of the key issues is the generator’s sensitivity to input data, which can cause sudden fluctuations in the generator’s loss value with certain inputs. This sensitivity suggests an inadequate ability to resist disturbances in the generator, causing the discriminator’s loss value to oscillate and negatively impacting the discriminator. Then, the negative feedback of discriminator is also not conducive to updating generator’s parameters, leading to suboptimal image generation quality. In response to this challenge, we present an innovative GANs model equipped with a learnable auxiliary module that processes auxiliary noise. The core objective of this module is to enhance the stability of both the generator and discriminator throughout the training process. To achieve this target, we incorporate a learnable auxiliary penalty and an augmented discriminator, designed to control the generator and reinforce the discriminator’s stability, respectively. We further apply our method to the Hinge and LSGANs loss functions, illustrating its efficacy in reducing the instability of both the generator and the discriminator. The tests we conducted on LSUN, CelebA, Market-1501 and Creative Senz3D datasets serve as proof of our method’s ability to improve the training stability and overall performance of the baseline methods.

computer science, information systems, theory & methods, software engineering

What problem does this paper attempt to address?

The paper primarily aims to address the training instability issues encountered by Generative Adversarial Networks (GANs) during the noise-to-image synthesis process. Specifically, the authors point out that the sensitivity of the generator to input data can lead to sudden fluctuations in the generator's loss value. These fluctuations result in insufficient robustness of the generator against interference, further affecting the stability of the discriminator and causing its loss value to fluctuate as well. The negative feedback from the discriminator hinders the effective updating of the generator's parameters, ultimately leading to poor image quality.

To tackle this challenge, the paper proposes an innovative GAN model with a learnable auxiliary module (GANs-LAM). This module enhances the stability of both the generator and the discriminator throughout the training process by handling auxiliary noise. Specifically, they introduce a learnable auxiliary penalty term and an enhanced discriminator design, with the former used to control the generator and the latter to reinforce the stability of the discriminator. Moreover, this approach is applied to the Hinge loss function and the LSGAN loss function, demonstrating its effectiveness in reducing instability between the generator and the discriminator. Experimental results show that this method improves the training stability and overall performance of benchmark methods on datasets such as LSUN, CelebA, Market-1501, and Creative Senz3D.