Abstract:Wasserstein-GANs have been introduced to address the deficiencies of generative adversarial networks (GANs) regarding the problems of vanishing gradients and mode collapse during the training, leading to improved convergence behaviour and improved image quality. However, Wasserstein-GANs require the discriminator to be Lipschitz continuous. In current state-of-the-art Wasserstein-GANs this constraint is enforced via gradient norm regularization. In this paper, we demonstrate that this regularization does not encourage a broad distribution of spectral-values in the discriminator weights, hence resulting in less fidelity in the learned distribution. We therefore investigate the possibility of substituting this Lipschitz constraint with an orthogonality constraint on the weight matrices. We compare three different weight orthogonalization techniques with regards to their convergence properties, their ability to ensure the Lipschitz condition and the achieved quality of the learned distribution. In addition, we provide a comparison to Wasserstein-GANs trained with current state-of-the-art methods, where we demonstrate the potential of solely using orthogonality-based regularization. In this context, we propose an improved training procedure for Wasserstein-GANs which utilizes orthogonalization to further increase its generalization capability. Finally, we provide a novel metric to evaluate the generalization capabilities of the discriminators of different Wasserstein-GANs.

Wasserstein Generative Adversarial Networks

Wasserstein generative adversarial networks

Some Theoretical Insights into Wasserstein GANs

Wasserstein Bounded Generative Adversarial Networks

Inferential Wasserstein generative adversarial networks

Wasserstein proximal of GANs

Relaxed Wasserstein with Applications to GANs

Wasserstein Generative Recurrent Adversarial Networks for Image Generating.

From GAN to WGAN

Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect.

Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect

A Wasserstein GAN model with the total variational regularization

Wasserstein-Wasserstein Auto-Encoders

Wasserstein Divergence for GANs

Sobolev Wasserstein GAN.

Orthogonal Wasserstein GANs

Quantum Wasserstein Generative Adversarial Networks.

Lp-WGAN: Using Lp-norm Normalization to Stabilize Wasserstein Generative Adversarial Networks

A Wasserstein perspective of Vanilla GANs

A Differential Equation Approach for Wasserstein GANs and Beyond