Abstract:With the resurgent interest in building open-domain dialogue systems, the dialogue generation task has attracted increasing attention over the past few years. This task is usually formulated as a conditional generation problem, which aims to generate a natural and meaningful response given dialogue contexts and specific constraints, such as persona. And maintaining a consistent persona is essential for the dialogue systems to gain trust from the users. Although tremendous advancements have been brought, traditional persona-based dialogue models are typically trained by leveraging a large number of persona-dense dialogue examples. Yet, such persona-dense training data are expensive to obtain, leading to a limited scale. This work presents a novel approach to learning from limited training examples by regarding consistency understanding as a regularization of response generation. To this end, we propose a novel stack-propagation framework for learning a generation and understanding <a class="link-external link-http" href="http://pipeline.Specifically" rel="external noopener nofollow">this http URL</a>, the framework stacks a Transformer encoder and two Transformer decoders, where the first decoder models response generation and the second serves as a regularizer and jointly models response generation and consistency understanding. The proposed framework can benefit from the stacked encoder and decoders to learn from much smaller personalized dialogue data while maintaining competitive performance. Under different low-resource settings, subjective and objective evaluations prove that the stack-propagation framework outperforms strong baselines in response quality and persona consistency and largely overcomes the shortcomings of traditional models that rely heavily on the persona-dense dialogue data.

PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable

Deep Reinforcement Learning for Dialogue Generation

Towards Efficient Dialogue Pre-training with Transferable and Interpretable Latent Structure

PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generation

A Unified Pre-training Framework for Conversational AI

DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation

PanGu-Bot: Efficient Generative Dialogue Pre-training from Pre-trained Language Model

Section-Aware Commonsense Knowledge-Grounded Dialogue Generation with Pre-trained Language Model.

A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Data

DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation

Knowledge-Grounded Dialogue Generation with Pre-trained Language Models

DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization

PlatoLM: Teaching LLMs in Multi-Round Dialogue via a User Simulator

GLM-Dialog: Noise-tolerant Pre-training for Knowledge-grounded Dialogue Generation

A Stack-Propagation Framework for Low-Resource Personalized Dialogue Generation

Pretrained Language Models for Dialogue Generation with Multiple Input Sources.

Video-Grounded Dialogues with Pretrained Generation Language Models

Multi-turn Response Selection with Commonsense-enhanced Language Models

An Empirical Investigation of Pre-Trained Transformer Language Models for Open-Domain Dialogue Generation

A New Dialogue Response Generation Agent for Large Language Models by Asking Questions to Detect User's Intentions

Response Generation in Longitudinal Dialogues: Which Knowledge Representation Helps?