Dialogue Strategy Adaptation to New Action Sets Using Multi-dimensional Modelling

Simon Keizer,Norbert Braunschweiler,Svetlana Stoyanchev,Rama Doddipatla

DOI: https://doi.org/10.1109/ASRU51503.2021.9688038

2022-04-15

Abstract:A major bottleneck for building statistical spoken dialogue systems for new domains and applications is the need for large amounts of training data. To address this problem, we adopt the multi-dimensional approach to dialogue management and evaluate its potential for transfer learning. Specifically, we exploit pre-trained task-independent policies to speed up training for an extended task-specific action set, in which the single summary action for requesting a slot is replaced by multiple slot-specific request actions. Policy optimisation and evaluation experiments using an agenda-based user simulator show that with limited training data, much better performance levels can be achieved when using the proposed multi-dimensional adaptation method. We confirm this improvement in a crowd-sourced human user evaluation of our spoken dialogue system, comparing partially trained policies. The multi-dimensional system (with adaptation on limited training data in the target scenario) outperforms the one-dimensional baseline (without adaptation on the same amount of training data) by 7% perceived success rate.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is how to reduce the need for a large amount of training data when building new dialogue system applications or expanding existing ones. Specifically, the paper adopts a multi - dimensional dialogue management method and evaluates its potential in transfer learning. The focus of the research is on using pre - trained task - independent policies to accelerate the training for extended task - specific action sets, especially replacing a single slot - request aggregation action with multiple slot - specific request actions. Through policy optimization and evaluation experiments using an agenda - based user simulator, and through crowdsourced human - user evaluations, the research shows that under limited training data, using the proposed multi - dimensional adaptation method can significantly improve performance levels.

Dialogue Strategy Adaptation to New Action Sets Using Multi-dimensional Modelling

Replicating Complex Dialogue Policy of Humans Via Offline Imitation Learning with Supervised Regularization.

Policy Adaptation for Deep Reinforcement Learning-Based Dialogue Management

Few-Shot Structured Policy Learning for Multi-Domain and Multi-Task Dialogues

Enabling Dialogue Management with Dynamically Created Dialogue Actions

Cross-domain Dialogue Policy Transfer via Simultaneous Speech-act and Slot Alignment

On-line policy optimisation of spoken dialogue systems via live interaction with human subjects

Leveraging Explicit Procedural Instructions for Data-Efficient Action Prediction

Policy Learning for Domain Selection in an Extensible Multi-domain Spoken Dialogue System

"Think Before You Speak": Improving Multi-Action Dialog Policy by Planning Single-Action Dialogs

Enhancing End-to-End Multi-Task Dialogue Systems: A Study on Intrinsic Motivation Reinforcement Learning Algorithms for Improved Training and Adaptability

Planning Like Human: A Dual-process Framework for Dialogue Planning

Prompt Learning for Domain Adaptation in Task-Oriented Dialogue

Domain-Specific Multi-Agent Dialog Policy Learning in Multi-Domain Task-Oriented Scenarios

Adversarial learning of neural user simulators for dialogue policy optimisation

Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents

Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems

Cooperative Multi-Agent Reinforcement Learning With Conversation Knowledge For Dialogue Management

Multi-Agent Task-Oriented Dialog Policy Learning with Role-Aware Reward Decomposition

Building Advanced Dialogue Managers for Goal-Oriented Dialogue Systems

Neural User Simulation for Corpus-based Policy Optimisation for Spoken Dialogue Systems