SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning

Chenyang Zhao,Xueying Jia,Vijay Viswanathan,Tongshuang Wu,Graham Neubig

2024-08-12

Abstract:Large language models (LLMs) hold the promise of solving diverse tasks when provided with appropriate natural language prompts. However, prompting often leads models to make predictions with lower accuracy compared to finetuning a model with ample training data. On the other hand, while finetuning LLMs on task-specific data generally improves their performance, abundant annotated datasets are not available for all tasks. Previous work has explored generating task-specific data from state-of-the-art LLMs and using this data to finetune smaller models, but this approach requires access to a language model other than the one being trained, which introduces cost, scalability challenges, and legal hurdles associated with continuously relying on more powerful LLMs. In response to these, we propose SELF-GUIDE, a multi-stage mechanism in which we synthesize task-specific input-output pairs from the student LLM, then use these input-output pairs to finetune the student LLM itself. In our empirical evaluation of the Natural Instructions V2 benchmark, we find that SELF-GUIDE improves the performance of LLM by a substantial margin. Specifically, we report an absolute improvement of approximately 15% for classification tasks and 18% for generation tasks in the benchmark's metrics. This sheds light on the promise of self-synthesized data guiding LLMs towards becoming task-specific experts without any external learning signals.

Computation and Language,Artificial Intelligence

What problem does this paper attempt to address?

The paper aims to address two main issues faced by large language models (LLMs) when handling specific tasks:

1. **Performance gap between Prompting and Fine-tuning**: Although LLMs can perform various tasks through natural language prompts, their accuracy is usually lower than methods that use a large amount of labeled data for fine-tuning. However, not all tasks have sufficient labeled data available.
2. **External dependency issue**: Previous research has attempted to use powerful "teacher" models to generate data for specific tasks to enhance smaller "student" models, but this requires continuous access to more powerful models, leading to cost, scalability, and legal obstacles.

To address these issues, the authors propose the SELF-GUIDE method, which fine-tunes the model through self-synthetic data to enhance its ability to perform specific tasks. Specifically, SELF-GUIDE generates synthetic input-output pairs based on a small amount of example data and then fine-tunes the model itself with this data. Experimental results show that SELF-GUIDE improves performance by approximately 14.5% in classification tasks and 17.9% in generation tasks. Additionally, SELF-GUIDE demonstrates its potential to guide LLMs to become experts in specific tasks under limited data conditions.