Abstract:Fine-tuning large language models (LLMs) on instruction datasets is a common way to improve their generative capabilities. However, instruction datasets can be expensive and time-consuming to manually curate, and while LLM-generated data is less labor-intensive, it may violate user privacy agreements or terms of service of LLM providers. Therefore, we seek a way of constructing instruction datasets with samples that are not generated by humans or LLMs but still improve LLM generative capabilities. In this work, we introduce Cookbook, a framework that programmatically generates training data consisting of simple patterns over random tokens, resulting in a scalable, cost-effective approach that avoids legal and privacy issues. First, Cookbook uses a template -- a data generating Python function -- to produce training data that encourages the model to learn an explicit pattern-based rule that corresponds to a desired task. We find that fine-tuning on Cookbook-generated data is able to improve performance on its corresponding task by up to 52.7 accuracy points. Second, since instruction datasets improve performance on multiple downstream tasks simultaneously, Cookbook algorithmically learns how to mix data from various templates to optimize performance on multiple tasks. On the standard multi-task GPT4ALL evaluation suite, Mistral-7B fine-tuned using a Cookbook-generated dataset attains the best accuracy on average compared to other 7B parameter instruction-tuned models and is the best performing model on 3 out of 8 tasks. Finally, we analyze when and why Cookbook improves performance and present a metric that allows us to verify that the improvement is largely explained by the model's generations adhering better to template rules.

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve The paper aims to address the issue of improving the generative capabilities of large language models (LLMs). Specifically, it focuses on how to enhance LLMs' generative abilities through programmatically generated datasets, which are neither manually created by humans nor generated by other LLMs. Traditional instruction datasets can significantly improve LLM performance, but their creation process is both time-consuming and expensive. Additionally, using user chat logs or LLM-generated data, while reducing manual costs, may violate user privacy agreements and the terms of service of LLM providers. Therefore, the paper proposes a new framework—**COOKBOOK**—to address these issues by programmatically generating training data. This data consists of simple patterns and random tokens, providing a scalable and cost-effective method that avoids legal and privacy concerns. COOKBOOK achieves this goal through the following two aspects: 1. **Single-Task Template Generation**: COOKBOOK uses templates (i.e., Python functions that generate data) to produce training data, encouraging the model to learn explicit pattern rules corresponding to specific tasks. Experimental results show that fine-tuning with data generated by COOKBOOK can significantly improve the model's performance on the corresponding tasks, with gains of up to 52.7 percentage points. 2. **Multi-Task Data Mixing**: Since instruction datasets can typically improve the performance of multiple downstream tasks simultaneously, COOKBOOK also proposes an algorithm to mix data from different templates to optimize performance across multiple tasks. In the standard multi-task GPT4ALL evaluation suite, the Mistral-7B model fine-tuned with COOKBOOK-generated datasets achieved the best average accuracy and performed optimally on 3 out of 8 tasks. ### Summary The main contribution of the paper is the proposal of a new framework, COOKBOOK, which improves the generative capabilities of LLMs through programmatically generated datasets, while avoiding the high costs and privacy issues associated with traditional methods. COOKBOOK excels not only in single-task performance but also demonstrates its advantages in multi-task scenarios.

Cookbook: A framework for improving LLM generative abilities via programmatic data generating templates

Cook-Gen: Robust Generative Modeling of Cooking Actions from Recipes

InstructCoder: Instruction Tuning Large Language Models for Code Editing

Harnessing the Power of David against Goliath: Exploring Instruction Data Generation without Using Closed-Source Models

StableLLaVA: Enhanced Visual Instruction Tuning with Synthesized Image-Dialogue Data

CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets

Dynamics of Instruction Tuning: Each Ability of Large Language Models Has Its Own Growth Pace

Training and Evaluating Language Models with Template-based Data Generation

Synthetic Data (Almost) from Scratch: Generalized Instruction Tuning for Language Models

FLawN-T5: An Empirical Examination of Effective Instruction-Tuning Data Mixtures for Legal Reasoning

IterSelectTune: An Iterative Training Framework for Efficient Instruction-Tuning Data Selection

Genixer: Empowering Multimodal Large Language Models as a Powerful Data Generator

Training Language Models on Synthetic Edit Sequences Improves Code Synthesis

Raw Text is All you Need: Knowledge-intensive Multi-turn Instruction Tuning for Large Language Model

Instruction Tuning with GPT-4

CodeLutra: Boosting LLM Code Generation via Preference-Guided Refinement

CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitions

GenQA: Generating Millions of Instructions from a Handful of Prompts

Towards Building the Federated GPT: Federated Instruction Tuning

AgentTuning: Enabling Generalized Agent Abilities for LLMs

Ensemble-Instruct: Generating Instruction-Tuning Data with a Heterogeneous Mixture of LMs