AnyTaskTune: Advanced Domain-Specific Solutions through Task-Fine-Tuning

Jiaxi Cui,Wentao Zhang,Jing Tang,Xudong Tong,Zhenwei Zhang,Amie,Jing Wen,Rongsheng Wang,Pengfei Wu

2024-07-10

Abstract:The pervasive deployment of Large Language Models-LLMs in various sectors often neglects the nuanced requirements of individuals and small organizations, who benefit more from models precisely tailored to their specific business contexts rather than those with broadly superior general capabilities. This work introduces \textbf{AnyTaskTune}, a novel fine-tuning methodology coined as \textbf{Task-Fine-Tune}, specifically developed to elevate model performance on a diverse array of domain-specific tasks. This method involves a meticulous process to identify and define targeted sub-tasks within a domain, followed by the creation of specialized enhancement datasets for fine-tuning, thereby optimizing task-specific model performance. We conducted comprehensive fine-tuning experiments not only in the legal domain for tasks such as keyword extraction and sentence prediction but across over twenty different sub-tasks derived from the domains of finance, healthcare, law, psychology, consumer services, and human resources. To substantiate our approach and facilitate community engagement, we will open-source these bilingual task datasets. Our findings demonstrate that models fine-tuned using the \textbf{Task-Fine-Tune} methodology not only achieve superior performance on these specific tasks but also significantly outperform models with higher general capabilities in their respective domains. Our work is publicly available at \url{<a class="link-external link-https" href="https://github.com/PandaVT/DataTager" rel="external noopener nofollow">this https URL</a>}.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The problem this paper attempts to address is that large language models (LLMs) often overlook the specific needs of individual users and small organizations in their widespread application across various fields. These users and organizations require models tailored to their specific business contexts rather than models with broad general capabilities. To this end, the paper introduces a new fine-tuning method—Task-Fine-Tune, which aims to optimize model performance on specific tasks by identifying and defining sub-tasks in particular domains and creating specialized augmented datasets for fine-tuning.

Specifically, the main contributions of the paper include:
1. **Introduction of the Task-Fine-Tune paradigm**: Specially designed for explicit datasets, emphasizing precision and specificity rather than general performance.
2. **Development and utilization of over 20 bilingual explicit datasets**: Covering multiple domains to validate the effectiveness of the Task-Fine-Tune method.
3. **Evaluation of multiple models and datasets**: Conducting experiments in different fields to demonstrate that task-specific fine-tuning outperforms traditional models.
4. **Open source code**: Providing the developed explicit datasets to offer resources for the community to further research and improve application-specific models.
5. **Implementation of a data processing model**: DataTager-LLM as an online accessible website, facilitating user access.

Through these methods, the paper demonstrates the significant advantages of Task-Fine-Tune in improving model performance on specific tasks, particularly in fields such as finance, healthcare, law, psychology, consumer services, and human resources.