Abstract:Incorporating external knowledge into the response generation process is essential to building more helpful and reliable dialog agents. However, collecting knowledge-grounded conversations is often costly, calling for a better pre-trained model for grounded dialog generation that generalizes well w.r.t. different types of knowledge. In this work, we propose KPT (Keyword-guided Pre-Training), a novel self-supervised pre-training method for grounded dialog generation without relying on extra knowledge annotation. Specifically, we use a pre-trained language model to extract the most uncertain tokens in the dialog as keywords. With these keywords, we construct two kinds of knowledge and pre-train a knowledge-grounded response generation model, aiming at handling two different scenarios: (1) the knowledge should be faithfully grounded; (2) it can be selectively used. For the former, the grounding knowledge consists of keywords extracted from the response. For the latter, the grounding knowledge is additionally augmented with keywords extracted from other utterances in the same dialog. Since the knowledge is extracted from the dialog itself, KPT can be easily performed on a large volume and variety of dialogue data. We considered three data sources (open-domain, task-oriented, conversational QA) with a total of 2.5M dialogues. We conduct extensive experiments on various few-shot knowledge-grounded generation tasks, including grounding on dialog acts, knowledge graphs, persona descriptions, and Wikipedia passages. Our comprehensive experiments and analyses demonstrate that KPT consistently outperforms state-of-the-art methods on these tasks with diverse grounding knowledge.

Training Two-Stage Knowledge-Grounded Dialogues with Attention Feedback

Read Key Points: Dialogue-Grounded Knowledge Points Generation with Multi-Level Salience-Aware Mixture

Reciprocal Learning of Knowledge Retriever and Response Ranker for Knowledge-Grounded Conversations.

Learning Multi-turn Response Selection in Grounded Dialogues with Reinforced Knowledge and Context Distillation

Dual-Feedback Knowledge Retrieval for Task-Oriented Dialogue Systems

CORE: Cooperative Training of Retriever-Reranker for Effective Dialogue Response Selection

Knowledge-Grounded Dialogue with Reward-Driven Knowledge Selection

Knowledge-Retrieval Task-Oriented Dialog Systems with Semi-Supervision

Improving knowledge-based dialogue generation through two-stage knowledge selection and knowledge selection-guided pointer network

Learning to Detect Relevant Contexts and Knowledge for Response Selection in Retrieval-based Dialogue Systems

GKS: Graph-based Knowledge Selector for Task-oriented Dialog System

Fine-grained Post-training for Improving Retrieval-based Dialogue Systems

Knowledge dialogue generation with multi-head attention mechanism

Knowledge-Grounded Dialogue Generation with Pre-trained Language Models

Knowledge Augmented Dialogue Generation with Divergent Facts Selection

HKA: A Hierarchical Knowledge Attention Mechanism for Multi-Turn Dialogue System

KRP-DS: A Knowledge Graph-Based Dialogue System with Inference-Aided Prediction

KPT: Keyword-guided Pre-training for Grounded Dialog Generation

Bridging the Gap Between Prior and Posterior Knowledge Selection for Knowledge-Grounded Dialogue Generation

Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning

Retrieval-Generation Alignment for End-to-End Task-Oriented Dialogue System