Abstract:Pre-training on large-scale open-domain dialogue data can substantially improve the performance of dialogue models. However, the pre-trained dialogue model's ability to utilize long-range context is limited due to the scarcity of long-turn dialogue sessions. Most dialogues in existing pre-training corpora contain fewer than three turns of dialogue. To alleviate this issue, we propose the Retrieve, Reorganize and Rescale framework (Re$^3$Dial), which can automatically construct billion-scale long-turn dialogues by reorganizing existing short-turn ones. Given a short-turn session, Re$^3$Dial first employs a session retriever to retrieve coherent consecutive sessions. To this end, we train the retriever to capture semantic and discourse relations within multi-turn dialogues through contrastive training. Next, Re$^3$Dial samples a session from retrieved results following a diversity sampling strategy, which is designed to penalize repetitive or generic sessions. A longer session is then derived by concatenating the original session and the sampled session. By repeating the above process, Re$^3$Dial can yield a coherent long-turn dialogue. Extensive experiments on multiple multi-turn dialogue benchmarks demonstrate that Re$^3$Dial significantly improves the dialogue model's ability to utilize long-range context and thus generate more sensible and informative responses. Finally, we build a toolkit for efficiently rescaling conversations with Re$^3$Dial, which enables us to construct a corpus containing 1B Chinese dialogue sessions with 11.3 turns on average (5$\times$ longer than the original corpus). Our retriever model, code, and data is publicly available at \url{<a class="link-external link-https" href="https://github.com/thu-coai/Re3Dial" rel="external noopener nofollow">this https URL</a>}.

C L ] 1 N ov 2 01 8 Dial 2 Desc : End-to-end Dialogue Description Generation

Dial2Desc: End-to-end Dialogue Description Generation.

Topic-Oriented Spoken Dialogue Summarization for Customer Service with Saliency-Aware Topic Modeling

Read Key Points: Dialogue-Grounded Knowledge Points Generation with Multi-Level Salience-Aware Mixture

DialogSum: A Real-Life Scenario Dialogue Summarization Dataset

JDDC 2.1: A Multimodal Chinese Dialogue Dataset with Joint Tasks of Query Rewriting, Response Generation, Discourse Parsing, and Summarization

Re$^3$Dial: Retrieve, Reorganize and Rescale Dialogue Corpus for Long-Turn Open-Domain Dialogue Pre-training

A Novel Topic Segmentation Approach for Enhanced Dialogue Summarization

DialogSum Challenge: Summarizing Real-Life Scenario Dialogues.

Re3Dial: Retrieve, Reorganize and Rescale Conversations for Long-Turn Open-Domain Dialogue Pre-training

Summarizing Dialogues with Negative Cues.

ClidSum: A Benchmark Dataset for Cross-Lingual Dialogue Summarization

DeepDial: Passage Completion on Dialogs.

Human-in-the-loop Abstractive Dialogue Summarization

Taxonomy of Abstractive Dialogue Summarization: Scenarios, Approaches, and Future Directions

A Survey on Dialogue Summarization: Recent Advances and New Frontiers

Abstractive Dialogue Summarization with Sentence-Gated Modeling Optimized by Dialogue Acts

A Benchmark for Understanding and Generating Dialogue between Characters in Stories

SuperDialseg: A Large-scale Dataset for Supervised Dialogue Segmentation

CSDS: A Fine-Grained Chinese Dataset for Customer Service Dialogue Summarization

An Exploratory Study on Long Dialogue Summarization: What Works and What's Next