Abstract:Generative adversarial nets (GANs) have generated a lot of excitement. Despite their popularity, they exhibit a number of well-documented issues in practice, which apparently contradict theoretical guarantees. A number of enlightening papers have pointed out that these issues arise from unjustified assumptions that are commonly made, but the message seems to have been lost amid the optimism of recent years. We believe the identified problems deserve more attention, and highlight the implications on both the properties of GANs and the trajectory of research on probabilistic models. We recently proposed an alternative method that sidesteps these problems.

What problem does this paper attempt to address?

This paper attempts to solve the problems encountered by Generative Adversarial Networks (GANs) in practical applications, which are in contradiction with theoretical guarantees. Specifically, the author points out the following key issues: 1. **Differences between the True Data Distribution and the Empirical Data Distribution**: - In theory, the training objective of GANs usually assumes access to the true data distribution \( p_{\text{data}} \), but in practice we can only sample from a limited training dataset. This leads to the use of the empirical data distribution \( \hat{p}_{\text{data}} \) to approximate the true data distribution. - Since the empirical data distribution is discrete while the model distribution is usually continuous, this can cause some metrics (such as Jensen - Shannon divergence and reverse KL divergence) to lose meaning or become unbounded. 2. **Problems of Asymptotic Consistency**: - The paper discusses the differences between the concept of asymptotic consistency used in GAN literature and the classical asymptotic consistency in statistics. For example, minimizing Jensen - Shannon divergence or reverse KL divergence does not mean that the parameter estimate is asymptotically consistent in the classical statistical sense. 3. **Mode Dropping Problem**: - A common problem with GANs is mode dropping, that is, the generator may ignore certain modes in the data distribution and focus only on some modes. This will affect the evaluation of the generative model because traditional maximum - likelihood - based methods will enforce complete recall, while GANs do not have such a limitation. 4. **Changes in Evaluation Metrics**: - Since GANs allow mode dropping, when evaluating generative models, not only precision but also recall needs to be considered. However, there is currently no reliable method to measure recall, which makes it unreliable to evaluate model performance solely based on sample quality. ### Solutions To solve the above problems, the author proposes a new method - **Implicit Maximum Likelihood Estimation (IMLE)**. IMLE aims to return to the principle of maximum likelihood and ensure that the model has complete recall. Specifically, IMLE is a parameter estimation technique that does not require explicit calculation of the likelihood function and is equivalent to maximum likelihood estimation under certain conditions. Through this method, the author hopes to promote the research of generative models in the direction of being able to learn the underlying data distribution, rather than just focusing on generating high - quality samples. ### Summary The main contribution of this paper lies in revealing some potential problems in the practical application of GANs and proposing an improved solution. The author emphasizes that understanding these implicit assumptions is crucial for correctly interpreting the theoretical results and practical applications of GANs.

On the Implicit Assumptions of GANs

Learning in Implicit Generative Models

On the Fairness of Generative Adversarial Networks (GANs)

Generative adversarial networks (GANs): Introduction, Taxonomy, Variants, Limitations, and Applications

The theoretical research of generative adversarial networks: an overview

Understanding GANs: fundamentals, variants, training challenges, applications, and open problems

Principled Knowledge Extrapolation with GANs.

A Mathematical Introduction to Generative Adversarial Nets (GAN)

Understanding GANs: the LQG Setting

Survey on Generative Adversarial Behavior in Artificial Neural Tasks

Latent Deep Space: Generative Adversarial Networks (GANs) in the Sciences

Game of GANs: Game-Theoretical Models for Generative Adversarial Networks

Understanding GANs in the LQG Setting: Formulation, Generalization and Stability

Convergence Problems with Generative Adversarial Networks (GANs)

Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-art

On the Anomalous Generalization of GANs

Uncertainty Principles Of Encoding Gans

A Review: Generative Adversarial Networks

The New Frontier of AI Research: Generative Adversarial Networks

Generative Adversarial Network: Some Analytical Perspectives