Dial-insight: Fine-tuning Large Language Models with High-Quality Domain-Specific Data Preventing Capability Collapse

Jianwei Sun,Chaoyang Mei,Linlin Wei,Kaiyu Zheng,Na Liu,Ming Cui,Tianyi Li

2024-03-14

Abstract:The efficacy of large language models (LLMs) is heavily dependent on the quality of the underlying data, particularly within specialized domains. A common challenge when fine-tuning LLMs for domain-specific applications is the potential degradation of the model's generalization capabilities. To address these issues, we propose a two-stage approach for the construction of production prompts designed to yield high-quality data. This method involves the generation of a diverse array of prompts that encompass a broad spectrum of tasks and exhibit a rich variety of expressions. Furthermore, we introduce a cost-effective, multi-dimensional quality assessment framework to ensure the integrity of the generated labeling data. Utilizing a dataset comprised of service provider and customer interactions from the real estate sector, we demonstrate a positive correlation between data quality and model performance. Notably, our findings indicate that the domain-specific proficiency of general LLMs can be enhanced through fine-tuning with data produced via our proposed method, without compromising their overall generalization abilities, even when exclusively domain-specific data is employed for fine-tuning.

Computation and Language

What problem does this paper attempt to address?

The paper aims to address the challenges faced by large language models (LLMs) during domain-specific fine-tuning, specifically how to enhance the model's domain expertise while maintaining its generalization ability. Specifically, the paper proposes a two-stage approach to generate high-quality data prompts and ensures the integrity of the generated labeled data through a multi-dimensional, cost-effective data quality evaluation framework. Experimental results show that the high-quality domain data generated by this method can significantly improve the professional capabilities of general LLMs without compromising their overall generalization performance, even when fine-tuning is performed using only domain-specific data.

The main contributions of the paper include:
1. **Complex Instruction Construction**: A two-stage evolutionary method is proposed to construct prompts containing domain information, role details, and task instructions. These prompts are more complex and richer than those generated by general instruction evolution algorithms.
2. **Comprehensive Data Quality Evaluation System**: A multi-dimensional, cost-effective data quality evaluation system is designed to objectively assess the quality of the entire training dataset, including richness, complexity, redundancy, and label quality.
3. **Maintaining Generalization Ability in Domain Fine-Tuning**: Unlike previous studies, the high-quality domain data generated by this method does not lead to a decline in the model's generalization ability. Even when fine-tuning is performed using only domain data, the model's basic generalization ability is maintained.