Abstract:With the recent exciting achievements of Google’s artificial intelligence system in the game of Go, deep reinforcement learning (DRL) has witnessed considerable development. DRL combines the abilities of sensing and making decisions provided by deep learning and reinforcement learning. Natural language processing (NLP) involves a large number of vocabularies or statements that have to be represented, and its subtasks, such as the dialogue system and machine translation, involve many decision problems that are difficult to model. Because of the aforementioned reasons, DRL can be appropriately applied to various NLP tasks such as named entity recognition, relation extraction, dialogue system, image caption, and machine translation. Further, DRL is helpful in improving the framework or the training pipeline of the aforementioned tasks, and notable achievements have been obtained. DRL is not an algorithm or a method but a paradigm. Many researchers fit plenty of NLP tasks in this paradigm and achieve better performance. Specifically, in text generation based on the reinforcement learning paradigm, the learning process that is used to produce a predicted sequence from the given source sequence can be considered to be the Markov decision process (MDP). In MDP, an agent interacts with the environment by receiving a sequence of observations and scaled rewards and subsequently produces the next action or word. This causes the text generation model to achieve decision-making ability, which can result in future success. Thus, the text generation task integrated with reinforcement learning is an attractive and promising research field. This study presented a comprehensive introduction and a systemic overview. First, we presented the basic methods in DRL and its variations. Then, we showed the main applications of DRL during the text generation task, trace the development of DRL, and summarized the merits and demerits associated with these applications. The final section enumerated some future research directions of DRL combined with NLP.

Deep Successor Feature Learning for Text Generation.

Deep Reinforcement Learning for Dialogue Generation

Deep Reinforcement Learning for NLP.

Learning to Decode for Future Success

Research Progress of Deep Reinforcement Learning Applied to Text Generation

Generating Text with Deep Reinforcement Learning

Successor Feature Neural Episodic Control

Learning Successor Features the Simple Way

FGGAN: Feature-Guiding Generative Adversarial Networks for Text Generation

Advantages and Limitations of using Successor Features for Transfer in Reinforcement Learning

Goal-directed Sequence Generation with Simulation Feedback Method

A Semi-Supervised Approach for Low-Resourced Text Generation.

Neural Text Generation: Past, Present and Beyond

Generative Cooperative Networks for Natural Language Generation

Optimizing Reinforcement Learning Using a Generative Action-Translator Transformer

Generative Adversarial Implicit Successor Representation

Text Feature Adversarial Learning for Text Generation With Knowledge Transfer From GPT2

Teacher Forcing Recovers Reward Functions for Text Generation

Goal-Directed Story Generation: Augmenting Generative Language Models with Reinforcement Learning

Efficient Deep Reinforcement Learning Via Policy-Extended Successor Feature Approximator

BoFGAN: Towards A New Structure of Backward-or-Forward Generative Adversarial Nets