Abstract:Multi-turn response selection is an important branch in the field of natural language processing, which aims to select the most appropriate response based on multi-turn dialogue. Most state-of-the-art models adopt pre-trained language models (PrLMs) and multiple auxiliary tasks to enhance their ability to understand the semantics in multi-turn dialogue. However, some critical challenges still remain to be addressed. Optimizing multiple auxiliary tasks simultaneously may significantly increase the training cost. Meanwhile, the semantic gap between the optimization objectives of the main and auxiliary tasks may bring noise to pre-trained language models. To address these challenges, we propose an efficient BERT-based neural network model with local context comprehension (BERT-LCC) for multi-turn response selection. First, we propose a self-supervised learning strategy, which introduces an auxiliary task named Response Prediction in Random Sliding Windows (RPRSW). In a multi-turn dialogue, the RPRSW task takes utterances falling within a random sliding window as input and predicts whether the last utterance within the sliding window is the appropriate response for the local dialogue context. This auxiliary task can enhance BERT’s understanding of local semantic information. Second, we propose a local information fusion (LIF) mechanism that collects multi-granularity local features at different dialogue stages and employs a gating function to fuse global features with local features. Third, we introduce a simple but effective domain learning strategy to learn rich semantic information at different dialogue stages during pre-training. Experimental results on two public benchmark datasets show that BERT-LCC outperforms other state-of-the-art models.

CAT-BERT: A Context-Aware Transferable BERT Model for Multi-turn Machine Reading Comprehension.

Improving Biomedical Named Entity Recognition with a Unified Multi-Task MRC Framework

Feeding What You Need by Understanding What You Learned

Enhancing Pre-Trained Language Representations with Rich Knowledge for Machine Reading Comprehension

A Multi-answer Multi-task Framework for Real-world Machine Reading Comprehension.

MMM: Multi-stage Multi-task Learning for Multi-choice Reading Comprehension

CA-BERT: Leveraging Context Awareness for Enhanced Multi-Turn Chat Interaction

A Multi-task Approach for Machine Reading Comprehension Form Named Entity Recognition Tasks

What does BERT Learn from Multiple-Choice Reading Comprehension Datasets?

Multi-Matching Network for Multiple Choice Reading Comprehension

Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond

A Multi-Task Learning Machine Reading Comprehension Model for Noisy Document (student Abstract)

Knowledge-Empowered Representation Learning for Chinese Medical Reading Comprehension: Task, Model and Resources

Improving BERT with local context comprehension for multi-turn response selection in retrieval-based dialogue systems

Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification.

MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding

Chinese Machine Reading Comprehension Based on Language Model Containing Knowledge.

Integrate Candidate Answer Extraction with Re-Ranking for Chinese Machine Reading Comprehension

Deep context modeling for multi-turn response selection in dialogue systems

A novel multi-domain machine reading comprehension model with domain interference mitigation

Named Entity Recognition via Machine Reading Comprehension: A Multi-Task Learning Approach