Abstract:Dialogue systems are a popular natural language processing (NLP) task as it is promising in real-life applications. It is also a complicated task since many NLP tasks deserving study are involved. As a result, a multitude of novel works on this task are carried out, and most of them are deep learning based due to their outstanding performance. In this survey, we mainly focus on the deep learning based dialogue systems. We comprehensively review state-of-the-art research outcomes in dialogue systems and analyze them from two angles: model type and system type. Specifically, from the angle of model type, we discuss the principles, characteristics, and applications of different models that are widely used in dialogue systems. This will help researchers acquaint these models and see how they are applied in state-of-the-art frameworks, which is rather helpful when designing a new dialogue system. From the angle of system type, we discuss task-oriented and open-domain dialogue systems as two streams of research, providing insight into the hot topics related. Furthermore, we comprehensively review the evaluation methods and datasets for dialogue systems to pave the way for future research. Finally, some possible research trends are identified based on the recent research outcomes. To the best of our knowledge, this survey is the most comprehensive and up-to-date one at present for deep learning based dialogue systems, extensively covering the popular techniques. We speculate that this work is a good starting point for academics who are new to the dialogue systems or those who want to quickly grasp up-to-date techniques in this area.

A Survey of Dialogue System Evaluation.

Survey of evaluation methods for dialogue systems}{Survey of evaluation methods for dialogue systems

Survey on Evaluation Methods for Dialogue Systems

How to Evaluate Your Dialogue Models: A Review of Approaches

Towards Unified Dialogue System Evaluation: A Comprehensive Analysis of Current Evaluation Protocols

On Dialogue Systems Based on Deep Learning

Don't Forget Your ABC's: Evaluating the State-of-the-Art in Chat-Oriented Dialogue Systems

Evaluating Task-oriented Dialogue Systems: A Systematic Review of Measures, Constructs and their Operationalisations

Recent advances in deep learning based dialogue systems: a systematic survey

How to Evaluate the Next System: Automatic Dialogue Evaluation from the Perspective of Continual Learning

A Review of Dialogue Systems: From Trained Monkeys to Stochastic Parrots

Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents

How NOT To Evaluate Your Dialogue System: An Empirical Study of Unsupervised Evaluation Metrics for Dialogue Response Generation

On Evaluating and Comparing Open Domain Dialog Systems

A review of dialogue systems: current trends and future directions

FFAEval: Evaluating Dialogue System Via Free-For-All Ranking

Dialogue Management Systems: a Survey and Overview

On the Use of Linguistic Features for the Evaluation of Generative Dialogue Systems

Medical Dialogue: A Survey of Categories, Methods, Evaluation and Challenges

Dialogue System: A Brief Review

A Survey on Learning-Based Approaches for Modeling and Classification of Human–Machine Dialog Systems