Abstract:Generative Adversarial Networks (GANs) have been widely applied in different scenarios thanks to the development of deep neural networks. The original GAN was proposed based on the non-parametric assumption of the infinite capacity of networks. However, it is still unknown whether GANs can fit the target distribution without any prior information. Due to the overconfident assumption, many issues remain unaddressed in GANs training, such as non-convergence, mode collapses, and gradient vanishing. Regularization and normalization are common methods of introducing prior information to stabilize training and improve discrimination. Although a handful number of regularization and normalization methods have been proposed for GANs, to the best of our knowledge, there exists no comprehensive survey that primarily focuses on objectives and development of these methods, apart from some incomprehensive and limited-scope studies. In this work, we conduct a comprehensive survey on the regularization and normalization techniques from different perspectives of GANs training. First, we systematically describe different perspectives of GANs training and thus obtain the different objectives of regularization and normalization. Based on these objectives, we propose a new taxonomy. Furthermore, we compare the performance of the mainstream methods on different datasets and investigate the applications of regularization and normalization techniques that have been frequently employed in state-of-the-art GANs. Finally, we highlight potential future directions of research in this domain. Code and studies related to the regularization and normalization of GANs in this work are summarized at https://github.com/iceli1007/GANs-Regularization-Review .

Regularization and Normalization for Generative Adversarial Networks: A Review

A Systematic Survey of Regularization and Normalization in GANs

Why Adopting Regularization and Normalization For Generative Adversarial Networks: A Survey

A Large-Scale Study on Regularization and Normalization in GANs

Regularization Methods for Generative Adversarial Networks: An Overview of Recent Studies

Stabilizing Training of Generative Adversarial Networks through Regularization

A Review: Generative Adversarial Networks

Direct Adversarial Training: A New Approach for Stabilizing The Training Process of GANs

CHAIN: Enhancing Generalization in Data-Efficient GANs via lipsCHitz continuity constrAIned Normalization

GraN-GAN: Piecewise Gradient Normalization for Generative Adversarial Networks

Consistency Regularization for Generative Adversarial Networks

Improving Generalization and Stability of Generative Adversarial Networks

Towards the Gradient Vanishing, Divergence Mismatching and Mode Collapse of Generative Adversarial Nets

Mode Regularized Generative Adversarial Networks

Towards a Better Understanding and Regularization of GAN Training Dynamics

On the Effects of Batch and Weight Normalization in Generative Adversarial Networks

Penalty Gradient Normalization for Generative Adversarial Networks

A Survey on Generative Adversarial Networks: Variants, Applications, and Training

The theoretical research of generative adversarial networks: an overview

Understanding GANs: fundamentals, variants, training challenges, applications, and open problems

A New Perspective on Stabilizing GANs Training: Direct Adversarial Training