Abstract:In standard generative adversarial network (SGAN), the discriminator estimates the probability that the input data is real. The generator is trained to increase the probability that fake data is real. We argue that it should also simultaneously decrease the probability that real data is real because 1) this would account for a priori knowledge that half of the data in the mini-batch is fake, 2) this would be observed with divergence minimization, and 3) in optimal settings, SGAN would be equivalent to integral probability metric (IPM) GANs. We show that this property can be induced by using a relativistic discriminator which estimate the probability that the given real data is more realistic than a randomly sampled fake data. We also present a variant in which the discriminator estimate the probability that the given real data is more realistic than fake data, on average. We generalize both approaches to non-standard GAN loss functions and we refer to them respectively as Relativistic GANs (RGANs) and Relativistic average GANs (RaGANs). We show that IPM-based GANs are a subset of RGANs which use the identity function. Empirically, we observe that 1) RGANs and RaGANs are significantly more stable and generate higher quality data samples than their non-relativistic counterparts, 2) Standard RaGAN with gradient penalty generate data of better quality than WGAN-GP while only requiring a single discriminator update per generator update (reducing the time taken for reaching the state-of-the-art by 400%), and 3) RaGANs are able to generate plausible high resolutions images (256x256) from a very small sample (N=2011), while GAN and LSGAN cannot; these images are of significantly better quality than the ones generated by WGAN-GP and SGAN with spectral normalization.

Reducing Differences Between Real and Realistic Samples to Improve GANs

SpatialGAN: Progressive Image Generation Based on Spatial Recursive Adversarial Expansion

DFS-GAN: stabilizing training of generative adversarial networks through discarding fake samples

Towards the Gradient Vanishing, Divergence Mismatching and Mode Collapse of Generative Adversarial Nets

D2PGGAN: Two Discriminators Used in Progressive Growing of GANS

Adversarial symmetric GANs: bridging adversarial samples and adversarial networks

Soft Generative Adversarial Network: Combating Mode Collapse in Generative Adversarial Network Training Via Dynamic Borderline Softening Mechanism

Simple Yet Effective Way for Improving the Performance of GAN

Incremental Focal Loss GANs.

Improving GANs with A Dynamic Discriminator

Improving GAN Training via Feature Space Shrinkage

Alleviation of Gradient Exploding in GANs: Fake Can Be Real

RDP-GAN: A Rényi-Differential Privacy Based Generative Adversarial Network

Soft-margin Ellipsoid generative adversarial networks

Improving GAN with Neighbors Embedding and Gradient Matching

SSD-GAN: Measuring the Realness in the Spatial and Spectral Domains

Dist-GAN: An Improved GAN using Distance Constraints

The relativistic discriminator: a key element missing from standard GAN

Alleviating Mode Collapse in GAN via Diversity Penalty Module

A New Perspective on Stabilizing GANs Training: Direct Adversarial Training

SMaRt: Improving GANs with Score Matching Regularity