Abstract:Multi-turn response selection is a major task in building intelligent dialogue systems. Most existing works focus on modeling the semantic relationship between the utterances and the candidate response with neural networks like RNNs and various attention mechanisms. In this paper, we study how to leverage the advantage of pre-trained language models (PTMs) to multi-turn response selection in retrieval-based chatbots. We propose a deep context modeling architecture (DCM) for multi-turn response selection by utilizing BERT as the context encoder. DCM is formulated as a four-module architecture, namely contextual encoder, utterance-to-response interaction, features aggregation, and response selection. Moreover, in DCM, we introduce the next utterance prediction as a pre-training scheme based on BERT, aiming to adapt general BERT to accommodate the inherent context continuity underlying the multi-turn dialogue. Taking BERT as the backbone encoder, we then investigate a variety of strategies to perform response selection with comprehensive comparisons. Empirical results on three public datasets from two different languages show that our proposed model outperforms existing promising models significantly, pushing recall to 86.8% (+5.2% improvement over BERT) on Ubuntu Dialogue corpus, recall to 68.5% (+6.4% improvement over BERT) on E-Commerce Dialogue corpus, MAP and MRR to 61.6% and 64.9% respectively (+2.3% and 1.8% improvement over BERT) on Douban Conversation corpus, achieving new state-of-the-art performance for multi-turn response selection.

Training Neural Response Selection for Task-Oriented Dialogue Systems

Exploring Dense Retrieval for Dialogue Response Selection

A Survey on Response Selection for Retrieval-based Dialogues.

CORE: Cooperative Training of Retriever-Reranker for Effective Dialogue Response Selection

Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues

RSVP: Customer Intent Detection via Agent Response Contrastive and Generative Pre-Training

Deep context modeling for multi-turn response selection in dialogue systems

Learning to Detect Relevant Contexts and Knowledge for Response Selection in Retrieval-based Dialogue Systems

Small Changes Make Big Differences: Improving Multi-turn Response Selection in Dialogue Systems Via Fine-Grained Contrastive Learning

DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation

Learning Multi-turn Response Selection in Grounded Dialogues with Reinforced Knowledge and Context Distillation

Fine-grained Post-training for Improving Retrieval-based Dialogue Systems

Different Strokes for Different Folks: Investigating Appropriate Further Pre-training Approaches for Diverse Dialogue Tasks

End-to-End Trainable Non-Collaborative Dialog System

Learning to Select Knowledge for Response Generation in Dialog Systems

A Network-based End-to-End Trainable Task-oriented Dialogue System

Towards Robust Online Dialogue Response Generation

Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning

Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbots

Pre-Trained and Attention-Based Neural Networks for Building Noetic Task-Oriented Dialogue Systems

Prompt Learning for Domain Adaptation in Task-Oriented Dialogue