Abstract:During multi-turn dialogue, with the increase in dialogue turns, the difficulty of intention recognition and the generation of the following sentence reply become more and more difficult. This paper mainly optimizes the context information extraction ability of the Seq2Seq Encoder in multi-turn dialogue modeling. We fuse the historical dialogue information and the current input statement information in the encoder to capture the context dialogue information better. Therefore, we propose a BERT-based fusion encoder ProBERT-To-GUR (PBTG) and an enhanced ELMO model 3-ELMO-Attention-GRU (3EAG). The two models mainly enhance the contextual information extraction capability of multi-turn dialogue. To verify the effectiveness of the two proposed models, we demonstrate the effectiveness of our model by combining data based on the LCCC-large multi-turn dialogue dataset and the Naturalconv multi-turn dataset. The experimental comparison results show that, in the multi-turn dialogue experiments of the open domain and fixed topic, the two Seq2Seq coding models proposed are significantly improved compared with the current state-of-the-art models. For specified topic multi-turn dialogue, the 3EAG model has the average BLEU value reaches the optimal 32.4, which achieves the best language generation effect, and the BLEU value in the actual dialogue verification experiment also surpasses 31.8. for open-domain multi-turn dialogue. The average BLEU value of the PBTG model reaches 31.8, the optimal 31.8 achieves the best language generation effect, and the BLEU value in the actual dialogue verification experiment surpasses 31.2. So, the 3EAG model is more suitable for fixed-topic multi-turn dialogues for the two tasks. The PBTG model is more muscular in open-domain multi-turn dialogue tasks; therefore, our model is significant for promoting multi-turn dialogue research.

Extending the Transformer with Context and Multi-dimensional Mechanism for Dialogue Response Generation.

Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network.

Multi-turn Response Selection using Dialogue Dependency Relations

Research on Task-oriented Dialogue Based on Modified Transformer

Multiturn dialogue generation by modeling sentence-level and discourse-level contexts

Human–Machine Multi-Turn Language Dialogue Interaction Based on Deep Learning

On Task-Level Dialogue Composition of Generative Transformer Model

Multi-turn Dialogue Generation Using Self-attention and Nonnegative Matrix Factorization

Enhancing Chinese Dialogue Generation with Word–Phrase Fusion Embedding and Sparse SoftMax Optimization

Deep context modeling for multi-turn response selection in dialogue systems

Filling the Gap of Utterance-aware and Speaker-aware Representation for Multi-turn Dialogue

Multi-turn Response Selection with Commonsense-enhanced Language Models

Transformer-Dw: A Transformer Network With Dynamic And Weighted Head

Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context.

A Hybrid Chinese Conversation Model Based on Retrieval and Generation.

Reasoning in Dialog: Improving Response Generation by Context Reading Comprehension

A Transformer-BERT Integrated Model-Based Automatic Conversation Method Under English Context

Learning Multi-Level Information for Dialogue Response Selection by Highway Recurrent Transformer

Modeling Multi-turn Conversation with Deep Utterance Aggregation

An Adaptive Contextual Relation Model for Improving Response Generation

Context-aware Memory Enhanced Transformer for End-to-end Task-Oriented Dialogue Systems