DiaSynth: Synthetic Dialogue Generation Framework for Low Resource Dialogue Applications

Sathya Krishnan Suresh,Wu Mengjun,Tushar Pranav,Eng Siong Chng

2024-10-15

Abstract:The scarcity of domain-specific dialogue datasets limits the development of dialogue systems across applications. Existing research is constrained by general or niche datasets that lack sufficient scale for training dialogue systems. To address this gap, we introduce DiaSynth - a synthetic dialogue generation framework capable of generating high-quality, contextually rich dialogues across a wide range of domains. Unlike existing frameworks, DiaSynth uses Large Language Models (LLMs) and Chain of Thought (CoT) reasoning to generate dynamic, domain-specific dialogues with simulated personas and diverse conversational features. We perform our experiments by generating synthetic data using different LLMs and few-shot examples from DialogSum and SAMSum. The pretrained language models fine-tuned on the synthetic data outperform the base models by 16.47% on dialogue summarization, while the comparison between models fine-tuned on in-domain data and synthetic data shows that the synthetic data is able to capture 90.48% of the performance distribution of the in-domain data on dialogue summarization. The quality of the data generated also increases as we increase the size of LLM from 3B to 8B. These results validate DiaSynth's potential as a robust alternative to traditional data collection methods. We open source the code and data generated for future research.

Computation and Language,Machine Learning

What problem does this paper attempt to address?

The problem this paper attempts to address is the scarcity of high-quality, large-scale, domain-specific dialogue datasets in the development of dialogue systems. Specifically, existing research is often limited by general or niche datasets, which are insufficient in scale and diversity to train effective dialogue systems. To bridge this gap, the paper proposes DiaSynth—a synthetic dialogue generation framework capable of generating high-quality, context-rich dialogues suitable for various domains. Unlike existing frameworks, DiaSynth leverages large language models (LLMs) and chain-of-thought (CoT) reasoning to generate dynamic, domain-specific dialogues and simulate different roles and diverse dialogue features.

The main contributions of the paper include:
1. **Generating high-quality dialogues**: DiaSynth generates high-quality, context-rich dialogues by simulating roles and dialogue features.
2. **Improving model performance**: Experimental results show that pre-trained language models fine-tuned with synthetic data generated by DiaSynth outperform baseline models in dialogue summarization tasks, with an average improvement of 16.47%.
3. **High data coverage**: The synthetic data captures 90.48% of the performance distribution of in-domain data, indicating its effectiveness in the absence of in-domain data.
4. **Scalability and flexibility**: The DiaSynth framework is highly scalable and flexible, capable of generating dialogue data of varying scales and diversity by adjusting model size and generation parameters.

Overall, DiaSynth provides a powerful and cost-effective alternative to address the data scarcity problem in dialogue system development.