Abstract:Task-oriented dialogue system is commonly formulated as a reinforcement learning problem. A reward served as a learning objective is offered at the end of the generated dialogue to help optimize the system. As fulfilling a specific task often takes many turns between the system and the user, a scalar reward signal after this long process can be delayed and sparse. To address the above problems in the reinforcement learning (RL) based task-completion system, we propose a novel hierarchical attentive adversarial network HaGAN which features a cascaded attentive generator CAG that explores a state-action space to generate a dialogue and global-local attentive discriminators GLAD to give a relevant reward at multi-scale dialogue states. Specifically, after every turn of the dialogue generation, the turn-based discriminator tests the current turn and give a local reward representing the generator's current generating ability. When the dialogue finishes, the dialogue-based discriminator gives a global reward concerns the whole dialog. Finally, a synthesized reward computed by combining global and local reward is returned to the generator. By doing so, the generator is able to generate globally and locally fluent and informative dialogues. Through experiments on two public benchmark datasets demonstrate the superiority of our HaGAN over other representative state-of-the-art methods.

Discourse-Aware Neural Rewards for Coherent Text Generation

Hagan: Hierarchical Attentive Adversarial Learning For Task-Oriented Dialogue System

Neural Net Models for Open-Domain Discourse Coherence

A Model of Coherence Based on Distributed Sentence Representation.

Deep Reinforcement Learning for Dialogue Generation

A Hierarchical Neural Autoencoder for Paragraphs and Documents

Modeling Coherence for Discourse Neural Machine Translation

Text Coherence Analysis Based on Deep Neural Network.

Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence

Learning to Write with Cooperative Discriminators

Leveraging Discourse Rewards for Document-Level Neural Machine Translation

Generative Cooperative Networks for Natural Language Generation

Natural Language Generation Using Reinforcement Learning with External Rewards

Neural Discourse Modeling of Conversations

Can Neural Generators for Dialogue Learn Sentence Planning and Discourse Structuring?

Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model

Analyzing Neural Discourse Coherence Models

Assessing Discourse Relations in Language Generation from GPT-2

Beyond Sparse Rewards: Enhancing Reinforcement Learning with Language Model Critique in Text Generation

Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Models

DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer