Abstract:Training generative models that can generate high-quality text with sufficient diversity is an important open problem for Natural Language Generation (NLG) community. Recently, generative adversarial models have been applied extensively on text generation tasks, where the adversarially trained generators alleviate the exposure bias experienced by conventional maximum likelihood approaches and result in promising generation quality. However, due to the notorious defect of mode collapse for adversarial training, the adversarially trained generators face a quality-diversity trade-off, i.e., the generator models tend to sacrifice generation diversity severely for increasing generation quality. In this paper, we propose a novel approach which aims to improve the performance of adversarial text generation via efficiently decelerating mode collapse of the adversarial training. To this end, we introduce a cooperative training paradigm, where a language model is cooperatively trained with the generator and we utilize the language model to efficiently shape the data distribution of the generator against mode collapse. Moreover, instead of engaging the cooperative update for the generator in a principled way, we formulate a meta learning mechanism, where the cooperative update to the generator serves as a high level meta task, with an intuition of ensuring the parameters of the generator after the adversarial update would stay resistant against mode collapse. In the experiment, we demonstrate our proposed approach can efficiently slow down the pace of mode collapse for the adversarial text generators. Overall, our proposed method is able to outperform the baseline approaches with significant margins in terms of both generation quality and diversity in the testified domains.

MetaEx-GAN: Meta Exploration to Improve Natural Language Generation Via Generative Adversarial Networks.

Composite Evolutionary GAN for Natural Language Generation with Temper Control

Generative Adversarial Exploration for Reinforcement Learning

MetaCGAN: A Novel GAN Model for Generating High Quality and Diversity Images with Few Training Data

Meta-CoTGAN: A Meta Cooperative Training Paradigm for Improving Adversarial Text Generation

Adversarial Ranking for Language Generation

Question Generation via Generative Adversarial Networks

Meta-Learning for Low-resource Natural Language Generation in Task-oriented Dialogue Systems.

Enhancing molecular design efficiency: Uniting language models and generative networks with genetic algorithms

ReGAN: RE[LAX|BAR|INFORCE] based Sequence Generation using GANs

OptAGAN: Entropy-based Finetuning on Text VAE-GAN

On Accurate Evaluation of GANs for Language Generation

GEGAN: Generative Adversarial Networks Training Based on Gradient Directed Evolutionary Strategy

Generative Cooperative Networks for Natural Language Generation

Unlocking the Power of GANs in Non-Autoregressive Text Generation

Language Modeling with Generative Adversarial Networks

What's New in My Data? Novelty Exploration via Contrastive Generation

Investigating the performance of Retrieval-Augmented Generation and fine-tuning for the development of AI-driven knowledge-based systems

Generative Adversarial Network Applications in Creating a Meta-Universe

Bidirectional Generative Adversarial Networks for Neural Machine Translation

BoFGAN: Towards A New Structure of Backward-or-Forward Generative Adversarial Nets