Instruction Tuning for Large Language Models: A Survey

Shengyu Zhang,Linfeng Dong,Xiaoya Li,Sen Zhang,Xiaofei Sun,Shuhe Wang,Jiwei Li,Runyi Hu,Tianwei Zhang,Fei Wu,Guoyin Wang

2024-03-14

Abstract:This paper surveys research works in the quickly advancing field of instruction tuning (IT), a crucial technique to enhance the capabilities and controllability of large language models (LLMs). Instruction tuning refers to the process of further training LLMs on a dataset consisting of \textsc{(instruction, output)} pairs in a supervised fashion, which bridges the gap between the next-word prediction objective of LLMs and the users' objective of having LLMs adhere to human instructions. In this work, we make a systematic review of the literature, including the general methodology of IT, the construction of IT datasets, the training of IT models, and applications to different modalities, domains and applications, along with an analysis on aspects that influence the outcome of IT (e.g., generation of instruction outputs, size of the instruction dataset, etc). We also review the potential pitfalls of IT along with criticism against it, along with efforts pointing out current deficiencies of existing strategies and suggest some avenues for fruitful research. Project page: <a class="link-external link-http" href="http://github.com/xiaoya-li/Instruction-Tuning-Survey" rel="external noopener nofollow">this http URL</a>

Computation and Language,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the mismatch between the training objectives of large - language models (LLMs) and user expectations. Specifically, LLMs are usually trained on large - scale corpora by minimizing the context - word prediction error, while users expect the model to "usefully and safely follow their instructions". To bridge this gap, the paper proposes the Instruction Tuning (IT) technique, which is a method for further training LLMs by using (instruction, output) pairs, aiming to improve the model's capabilities and controllability.

The main contribution of the paper lies in systematically reviewing the literature in the IT field, including the general methodology of IT, the construction of IT datasets, the training of IT models, and their applications in different modalities, domains, and applications, and analyzing the factors that affect the IT effect (such as the generation of instruction outputs, the size of the instruction dataset, etc.). In addition, the paper also discusses the potential problems of IT and its criticisms, points out the deficiencies of existing strategies, and proposes some directions for future research.