Abstract:Visual Dialogue task requires an agent to be engaged in a conversation with human about an image. The ability of generating detailed and non-repetitive responses is crucial for the agent to achieve human-like conversation. In this paper, we propose a novel generative decoding architecture to generate high-quality responses, which moves away from decoding the whole encoded semantics towards the design that advocates both transparency and flexibility. In this architecture, word generation is decomposed into a series of attention-based information selection steps, performed by the novel recurrent Deliberation, Abandon and Memory (DAM) module. Each DAM module performs an adaptive combination of the response-level semantics captured from the encoder and the word-level semantics specifically selected for generating each word. Therefore, the responses contain more detailed and non-repetitive descriptions while maintaining the semantic accuracy. Furthermore, DAM is flexible to cooperate with existing visual dialogue encoders and adaptive to the encoder structures by constraining the information selection mode in DAM. We apply DAM to three typical encoders and verify the performance on the VisDial v1.0 dataset. Experimental results show that the proposed models achieve new state-of-the-art performance with high-quality responses. The code is available at https://github.com/JXZe/DAM.

HL-EncDec: A Hybrid-Level Encoder-Decoder for Neural Response Generation.

DLCEncDec : A Fully Character-Level Encoder-Decoder Model for Neural Responding Conversation.

A Simple, Fast Diverse Decoding Algorithm for Neural Generation

Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints

Learning to Decode for Future Success

MEMD: A Diversity-Promoting Learning Framework for Short-Text Conversation.

Neural Response Generation with Dynamic Vocabularies

Cue-word Driven Neural Response Generation with a Shrinking Vocabulary

Neural Response Generation with Meta-Words

Neural Response Generation via GAN with an Approximate Embedding Layer.

DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation

A Simple Dual-decoder Model for Generating Response with Sentiment

DAM: Deliberation, Abandon and Memory Networks for Generating Detailed and Non-repetitive Responses in Visual Dialogue

A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues

Affective Neural Response Generation

A Character-Level Decoder without Explicit Segmentation for Neural Machine Translation

Improving Open-Ended Text Generation via Adaptive Decoding

HSCJN: A Holistic Semantic Constraint Joint Network for Diverse Response Generation

Modeling Extractive Question Answering Using Encoder-Decoder Models with Constrained Decoding and Evaluation-Based Reinforcement Learning

Group Linguistic Bias Aware Neural Response Generation.

Multi-channel Encoder for Neural Machine Translation