Abstract:Conditional variational models, using either continuous or discrete latent variables, are powerful for open-domain dialogue response generation. However, previous works show that continuous latent variables tend to reduce the coherence of generated responses. In this paper, we also found that discrete latent variables have difficulty capturing more diverse expressions. To tackle these problems, we combine the merits of both continuous and discrete latent variables and propose a Hybrid Latent Variable (HLV) method. Specifically, HLV constrains the global semantics of responses through discrete latent variables and enriches responses with continuous latent variables. Thus, we diversify the generated responses while maintaining relevance and coherence. In addition, we propose Conditional Hybrid Variational Transformer (CHVT) to construct and to utilize HLV with transformers for dialogue generation. Through fine-grained symbolic-level semantic information and additive Gaussian mixing, we construct the distribution of continuous variables, prompting the generation of diverse expressions. Meanwhile, to maintain the relevance and coherence, the discrete latent variable is optimized by self-separation training. Experimental results on two dialogue generation datasets (DailyDialog and Opensubtitles) show that CHVT is superior to traditional transformer-based variational mechanism w.r.t. diversity, relevance and coherence metrics. Moreover, we also demonstrate the benefit of applying HLV to fine-tuning two pre-trained dialogue models (PLATO and BART-base).

Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoders

A Simple, Fast Diverse Decoding Algorithm for Neural Generation

Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints

Deep Reinforcement Learning for Dialogue Generation

Adversarial Learning for Neural Dialogue Generation.

DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation

Improving the Responses Diversity of Persona-based Neural Conversation System: A Conditional Variational Autoencoders-based Approach

A Latent-Constrained Variational Neural Dialogue Model for Information-Rich Responses

MEMD: A Diversity-Promoting Learning Framework for Short-Text Conversation.

Dirichlet Latent Variable Hierarchical Recurrent Encoder-Decoder in Dialogue Generation.

DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder

A Discrete CVAE for Response Generation on Short-Text Conversation.

Advanced Conditional Variational Autoencoders (A-CVAE): Towards interpreting open-domain conversation generation via disentangling latent feature representation

Towards Diverse, Relevant and Coherent Open-Domain Dialogue Generation Via Hybrid Latent Variables.

A Novel Two-Stage Generation Framework for Promoting the Persona-Consistency and Diversity of Responses in Neural Dialog Systems

Variational Autoregressive Decoder for Neural Response Generation.

Hierarchy Response Learning for Neural Conversation Generation

Emotional Dialogue Generation Based on Conditional Variational Autoencoder and Dual Emotion Framework

A Hierarchical Latent Structure for Variational Conversation Modeling

Towards a Neural Conversation Model With Diversity Net Using Determinantal Point Processes.

Generating Relevant and Coherent Dialogue Responses Using Self-separated Conditional Variational AutoEncoders