Abstract:Wasserstein-GANs have been introduced to address the deficiencies of generative adversarial networks (GANs) regarding the problems of vanishing gradients and mode collapse during the training, leading to improved convergence behaviour and improved image quality. However, Wasserstein-GANs require the discriminator to be Lipschitz continuous. In current state-of-the-art Wasserstein-GANs this constraint is enforced via gradient norm regularization. In this paper, we demonstrate that this regularization does not encourage a broad distribution of spectral-values in the discriminator weights, hence resulting in less fidelity in the learned distribution. We therefore investigate the possibility of substituting this Lipschitz constraint with an orthogonality constraint on the weight matrices. We compare three different weight orthogonalization techniques with regards to their convergence properties, their ability to ensure the Lipschitz condition and the achieved quality of the learned distribution. In addition, we provide a comparison to Wasserstein-GANs trained with current state-of-the-art methods, where we demonstrate the potential of solely using orthogonality-based regularization. In this context, we propose an improved training procedure for Wasserstein-GANs which utilizes orthogonalization to further increase its generalization capability. Finally, we provide a novel metric to evaluate the generalization capabilities of the discriminators of different Wasserstein-GANs.

Wasserstein generative adversarial networks

Wasserstein Generative Adversarial Networks

Wasserstein Bounded Generative Adversarial Networks

Some Theoretical Insights into Wasserstein GANs

Inferential Wasserstein generative adversarial networks

Wasserstein proximal of GANs

From GAN to WGAN

Relaxed Wasserstein with Applications to GANs

Wasserstein Generative Recurrent Adversarial Networks for Image Generating.

A Wasserstein GAN model with the total variational regularization

Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect

Orthogonal Wasserstein GANs

Improving the Improved Training of Wasserstein GANs: A Consistency Term and Its Dual Effect.

A Wasserstein perspective of Vanilla GANs

Wasserstein-Wasserstein Auto-Encoders

Wasserstein Divergence for GANs

Optimal 1-Wasserstein Distance for WGANs

Lp-WGAN: Using Lp-norm Normalization to Stabilize Wasserstein Generative Adversarial Networks

GraphWGAN: Graph Representation Learning with Wasserstein Generative Adversarial Networks

A Differential Equation Approach for Wasserstein GANs and Beyond