Abstract:Multi-turn response selection is a major task in building intelligent dialogue systems. Most existing works focus on modeling the semantic relationship between the utterances and the candidate response with neural networks like RNNs and various attention mechanisms. In this paper, we study how to leverage the advantage of pre-trained language models (PTMs) to multi-turn response selection in retrieval-based chatbots. We propose a deep context modeling architecture (DCM) for multi-turn response selection by utilizing BERT as the context encoder. DCM is formulated as a four-module architecture, namely contextual encoder, utterance-to-response interaction, features aggregation, and response selection. Moreover, in DCM, we introduce the next utterance prediction as a pre-training scheme based on BERT, aiming to adapt general BERT to accommodate the inherent context continuity underlying the multi-turn dialogue. Taking BERT as the backbone encoder, we then investigate a variety of strategies to perform response selection with comprehensive comparisons. Empirical results on three public datasets from two different languages show that our proposed model outperforms existing promising models significantly, pushing recall to 86.8% (+5.2% improvement over BERT) on Ubuntu Dialogue corpus, recall to 68.5% (+6.4% improvement over BERT) on E-Commerce Dialogue corpus, MAP and MRR to 61.6% and 64.9% respectively (+2.3% and 1.8% improvement over BERT) on Douban Conversation corpus, achieving new state-of-the-art performance for multi-turn response selection.

Attention Based Dialogue Context Selection Model

Deep context modeling for multi-turn response selection in dialogue systems

Improving Contextual Language Models for Response Retrieval in Multi-Turn Conversation

Dynamically Context-sensitive Time-decay Attention for Dialogue Modeling

How to Represent Context Better? an Empirical Study on Context Modeling for Multi-turn Response Selection.

Local Contextual Attention with Hierarchical Structure for Dialogue Act Recognition

Dynamic Time-Aware Attention to Speaker Roles and Contexts for Spoken Language Understanding

Evaluating and Improving Context Attention Distribution on Multi-Turn Response Generation using Self-Contained Distractions

Dialogue Generation Model with Hierarchical Encoding and Semantic Segmentation of Dialogue Context

Topic-Aware Multi-turn Dialogue Modeling

Channel-Aware Decoupling Network for Multiturn Dialog Comprehension.

Contextualized Attention-based Knowledge Transfer for Spoken Conversational Question Answering

Context Directed Speech Recognition in Dialogue Systems

Improve Retrieval-based Dialogue System via Syntax-Informed Attention

How To Make Context More Useful? An Empirical Study On Context-Aware Neural Conversational Models

Exploring Dense Retrieval for Dialogue Response Selection

A Survey on Response Selection for Retrieval-based Dialogues.

A Static and Dynamic Attention Framework for Multi Turn Dialogue Generation

Learning to Detect Relevant Contexts and Knowledge for Response Selection in Retrieval-based Dialogue Systems

Leveraging Neural Network-Based Model for Context Classification of Classroom Dialogue Text

Small Changes Make Big Differences: Improving Multi-turn Response Selection in Dialogue Systems Via Fine-Grained Contrastive Learning