Abstract:Instruction tuning is instrumental in enabling Large Language Models~(LLMs) to follow user instructions to complete various open-domain tasks. The success of instruction tuning depends on the availability of high-quality instruction data. Owing to the exorbitant cost and substandard quality of human annotation, recent works have been deeply engaged in the exploration of the utilization of powerful closed-source models to generate instruction data automatically. However, these methods carry potential risks arising from the usage requirements of powerful closed-source models, which strictly forbid the utilization of their outputs to develop machine learning models. To deal with this problem, in this work, we explore alternative approaches to generate high-quality instruction data that do not rely on closed-source models. Our exploration includes an investigation of various existing instruction generation methods, culminating in the integration of the most efficient variant with two novel strategies to enhance the quality further. Evaluation results from two benchmarks and the GPT-4 model demonstrate the effectiveness of our generated instruction data, which can outperform Alpaca, a method reliant on closed-source models. We hope that more progress can be achieved in generating high-quality instruction data without using closed-source models.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to generate high - quality instruction data without using closed - source models. Specifically, due to the limitations in the use of closed - source models (for example, it is prohibited to use their outputs to develop machine - learning models), as well as the high cost of manual annotation and low quality, the author explores the method of using open - source models to automatically generate high - quality instruction data. The paper improves the quality and diversity of generated instruction data by studying existing instruction generation methods and combining two new strategies - the instruction filtering strategy and the extraction - generation strategy. Eventually, the paper proposes a new framework that can generate higher - quality instruction data than existing methods (including methods relying on closed - source models) without relying on closed - source models. The main contributions of the paper include: 1. Exploring the possibility of generating instruction data without using closed - source models and studying different variants of existing instruction data generation methods. 2. Proposing a new instruction filtering strategy to ensure the alignment between the generated instructions and the output. 3. Introducing an extraction - generation strategy to generate more diverse instructions by extracting diverse fragments from existing texts. 4. Verifying the effectiveness of the generated data through the evaluation results of two benchmark tests and the GPT - 4 model, showing the potential of open - source models in generating high - quality instruction data, and its performance can be comparable to the data generated using closed - source models.

Harnessing the Power of David against Goliath: Exploring Instruction Data Generation without Using Closed-Source Models

Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models

TeGit: Generating High-Quality Instruction-Tuning Data with Text-Grounded Task Design

Instruction Tuning with GPT-4

Dynosaur: A Dynamic Growth Paradigm for Instruction-Tuning Data Curation

InverseCoder: Self-improving Instruction-Tuned Code LLMs with Inverse-Instruct

Explore-Instruct: Enhancing Domain-Specific Instruction Coverage through Active Exploration

Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models

Optimizing Instruction Synthesis: Effective Exploration of Evolutionary Space with Tree Search

DoG-Instruct: Towards Premium Instruction-Tuning Data Via Text-Grounded Instruction Wrapping

Non-instructional Fine-tuning: Enabling Instruction-Following Capabilities in Pre-trained Language Models without Instruction-Following Data

GenQA: Generating Millions of Instructions from a Handful of Prompts

INSTRUCTEVAL: Towards Holistic Evaluation of Instruction-Tuned Large Language Models

Instruction Mining: Instruction Data Selection for Tuning Large Language Models

Dynamics of Instruction Tuning: Each Ability of Large Language Models Has Its Own Growth Pace

A High-Quality Text-Rich Image Instruction Tuning Dataset via Hybrid Instruction Generation

Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor

Rapidly Developing High-quality Instruction Data and Evaluation Benchmark for Large Language Models with Minimal Human Effort: A Case Study on Japanese

EasyInstruct: An Easy-to-use Instruction Processing Framework for Large Language Models

REInstruct: Building Instruction Data from Unlabeled Corpus

Towards Robust Instruction Tuning on Multimodal Large Language Models