Abstract:Human conversations naturally evolve around related concepts and scatter to multi-hop concepts. This paper presents a new conversation generation model, ConceptFlow, which leverages commonsense knowledge graphs to explicitly model conversation flows. By grounding conversations to the concept space, ConceptFlow represents the potential conversation flow as traverses in the concept space along commonsense relations. The traverse is guided by graph attentions in the concept graph, moving towards more meaningful directions in the concept space, in order to generate more semantic and informative responses. Experiments on Reddit conversations demonstrate ConceptFlow's effectiveness over previous knowledge-aware conversation models and GPT-2 based models while using 70% fewer parameters, confirming the advantage of explicit modeling conversation structures. All source codes of this work are available at <a class="link-external link-https" href="https://github.com/thunlp/ConceptFlow" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

### Problems the paper attempts to solve This paper aims to solve several common problems in dialogue generation models: 1. **Monotonicity and repetitiveness of generated content**: Traditional dialogue generation models (such as Seq2Seq - based models) are prone to generate monotonous, repetitive and meaningless responses when generating dialogues. These problems often cause the responses of dialogue assistants to deviate from the topic and lack information. 2. **Lack of explicit modeling of dialogue structure**: Human conversations usually revolve around related concepts and gradually shift to multi - step related concepts. However, existing dialogue generation models fail to well capture the changes in this dialogue structure, resulting in generated dialogues that are not natural and coherent enough. 3. **Insufficient effective use of external knowledge**: Although some studies attempt to improve dialogue quality by introducing external knowledge (such as knowledge graphs or common - sense databases), these methods are usually limited to the knowledge that directly appears in the dialogue and fail to fully utilize remotely related concepts. To solve the above problems, the paper proposes a new dialogue generation model - **ConceptFlow**. This model improves dialogue generation in the following ways: - **Explicitly modeling dialogue flow using common - sense knowledge graphs**: ConceptFlow regards dialogue as a traversal process in a common - sense knowledge graph and guides the dialogue to develop in a more meaningful direction through the graph attention mechanism. - **Expanding to multi - step related concepts**: ConceptFlow not only considers directly related concepts (zero - hop and one - hop concepts), but also expands to more distant concepts (two - hop concepts) to better simulate the concept shift in natural dialogues. - **Improving the quality and diversity of generated responses**: By explicitly modeling the dialogue structure, ConceptFlow can generate more informative, more relevant and less repetitive dialogue responses. ### Experimental results Experiments show that ConceptFlow is significantly superior to existing models in multiple aspects: - **Automatic evaluation**: ConceptFlow performs well in multiple metrics such as relevance, diversity and novelty, especially leading by a large margin in metrics such as BLEU and ROUGE. - **Human evaluation**: The results of human evaluation show that the dialogues generated by ConceptFlow are superior to other models in terms of appropriateness and information content, and the number of parameters is only 30% of that of GPT - 2. - **Effectiveness of multi - step concepts**: Experiments verify the importance of the selection of multi - step concepts for dialogue generation and prove that the long - distance concepts selected by ConceptFlow through the graph attention mechanism can effectively improve dialogue quality. In summary, ConceptFlow successfully solves the problems existing in traditional dialogue generation models by explicitly modeling dialogue flow and using multi - step related concepts, and significantly improves the performance of dialogue systems.

Grounded Conversation Generation as Guided Traverses in Commonsense Knowledge Graphs

Enhancing Dialogue Generation with Conversational Concept Flows

Commonsense Knowledge Aware Conversation Generation with Graph Attention

Guiding Topic Flows in the Generative Chatbot by Enhancing the ConceptNet with the Conversation Corpora

Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph

C3KG: A Chinese Commonsense Conversation Knowledge Graph

Conversational Graph Grounded Policy Learning for Open-Domain Conversation Generation.

Decoder Encoder Attention Knowledge Interpreter Knowledge Aware Generator Static Graph Attention Dynamic Graph Attention

Knowledge Graph Grounded Goal Planning for Open-Domain Conversation Generation

Knowledge Aware Conversation Generation with Explainable Reasoning over Augmented Graphs

Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering

Generative commonsense knowledge subgraph retrieval for open-domain dialogue response generation

Commonsense Generative Model for Chinese Automatic Knowledge Graph Construction

Select, Extend, and Generate: Generative Knowledge Selection for Open-Domain Dialogue Response Generation.

Interconnected Question Generation with Coreference Alignment and Conversation Flow Modeling

Knowledge Aware Conversation Generation with Reasoning on Augmented Graph

KGR^4: Retrieval, Retrospect, Refine and Rethink for Commonsense Generation

Building knowledge-grounded dialogue systems with graph-based semantic modeling

Knowledge-Grounded Conversational Data Augmentation with Generative Conversational Networks

A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation