Abstract:Dialogue policy is a crucial component in task-oriented Spoken Dialogue Systems (SDSs). As a decision function, it takes the current dialogue state as input and generates appropriate system’s response. In this paper, we explore the reinforcement learning approaches to solve this problem in an Indic language scenario. Recently, Deep Reinforcement Learning (DRL) has been used to optimise the dialogue policy. However, many DRL approaches are not sample-efficient. Hence, particular attention is given to actor-critic methods based on off-policy reinforcement learning that utilise the Experience Replay (ER) technique for reducing the bias and variance to achieve high sample efficiency. ER based actor-critic methods, such as Advantage Actor-Critic Experience Replay (A2CER) are proven to deliver competitive results in gaming environments that are fully observable and have a very small action-set. While, in SDSs, the states are not fully observable and often have to deal with the large action space. Describing the limitations of traditional methods, i.e., value-based and policy-based methods, such as high variance, low sample-efficiency, and often converging to local optima, we firstly explore the use of A2CER in dialogue policy learning. It is shown to beat the current state-of-the-art deep learning methods for SDS. Secondly, to handle the issues of early-stage performance, we utilise a demonstration corpus to pre-train the models prior to on-line policy learning. We thus experiment with the A2CER on a larger action space and find it significantly faster than the current state-of-the-art. Combining both approaches, we present a novel DRL based dialogue policy optimisation method, A2CER and its effectiveness for a task-oriented SDS in the Indic language.

Distributed Structured Actor-Critic Reinforcement Learning for Universal Dialogue Management

Deep Reinforcement Learning for Dialogue Generation

AgentGraph: Towards Universal Dialogue Management with Structured Deep Reinforcement Learning

AgentGraph: Toward Universal Dialogue Management with Structured Deep Reinforcement Learning

Experience Replay-based Deep Reinforcement Learning for Dialogue Management Optimisation

Task-oriented Dialogue System Based on Reinforcement Learning

Sample-efficient Actor-Critic Reinforcement Learning with Supervised Data for Dialogue Management

Offline Reinforcement Learning for Mixture-of-Expert Dialogue Management

Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems

Deep Reinforcement Learning for On-line Dialogue State Tracking

Composite Task-Completion Dialogue Policy Learning via Hierarchical Deep Reinforcement Learning

Structured Dialogue Policy with Graph Neural Networks.

Cooperative Multi-Agent Reinforcement Learning With Conversation Knowledge For Dialogue Management

Rewarding What Matters: Step-by-Step Reinforcement Learning for Task-Oriented Dialogue

Feudal Reinforcement Learning for Dialogue Management in Large Domains

Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning.

Cascaded LSTMs based Deep Reinforcement Learning for Goal-driven Dialogue

Word-Based POMDP Dialog Management Via Hybrid Learning.

Reinforcement Learning for Personalized Dialogue Management

End-to-End Optimization of Task-Oriented Dialogue Model with Deep Reinforcement Learning

Multi-Agent Task-Oriented Dialog Policy Learning with Role-Aware Reward Decomposition