Abstract:Large language models (LLMs) have demonstrated remarkable capabilities in various tasks. However, their suitability for domain-specific tasks, is limited due to their immense scale at deployment, susceptibility to misinformation, and more importantly, high data annotation costs. We propose a novel Interactive Multi-Fidelity Learning (IMFL) framework for the cost-effective development of small domain-specific LMs under limited annotation budgets. Our approach formulates the domain-specific fine-tuning process as a multi-fidelity learning problem, focusing on identifying the optimal acquisition strategy that balances between low-fidelity automatic LLM annotations and high-fidelity human annotations to maximize model performance. We further propose an exploration-exploitation query strategy that enhances annotation diversity and informativeness, incorporating two innovative designs: 1) prompt retrieval that selects in-context examples from human-annotated samples to improve LLM annotation, and 2) variable batch size that controls the order for choosing each fidelity to facilitate knowledge distillation, ultimately enhancing annotation quality. Extensive experiments on financial and medical tasks demonstrate that IMFL achieves superior performance compared with single fidelity annotations. Given a limited budget of human annotation, IMFL significantly outperforms the human annotation baselines in all four tasks and achieves very close performance as human annotations on two of the tasks. These promising results suggest that the high human annotation costs in domain-specific tasks can be significantly reduced by employing IMFL, which utilizes fewer human annotations, supplemented with cheaper and faster LLM (e.g., GPT-3.5) annotations to achieve comparable performance.

Multi-view Fusion for Instruction Mining of Large Language Model

Instruction Mining: Instruction Data Selection for Tuning Large Language Models

Align^2LLaVA: Cascaded Human and Large Language Model Preference Alignment for Multi-modal Instruction Curation

Maybe Only 0.5 Training Data Instruction Tuning

EasyInstruct: An Easy-to-use Instruction Processing Framework for Large Language Models

Improving Multilingual Instruction Finetuning via Linguistically Natural and Diverse Datasets

Towards Robust Instruction Tuning on Multimodal Large Language Models

Align$^2$LLaVA: Cascaded Human and Large Language Model Preference Alignment for Multi-modal Instruction Curation

Demystifying Instruction Mixing for Fine-tuning Large Language Models

Explore-Instruct: Enhancing Domain-Specific Instruction Coverage through Active Exploration

Rethinking the Instruction Quality: LIFT is What You Need

Multimodal Instruction Tuning with Conditional Mixture of LoRA

BioInstruct: Instruction Tuning of Large Language Models for Biomedical Natural Language Processing

From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning

MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity

Interactive Multi-fidelity Learning for Cost-effective Adaptation of Language Model with Sparse Human Supervision

Multilingual Instruction Tuning With Just a Pinch of Multilinguality

From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning

Beyond IID: Optimizing Instruction Learning from the Perspective of Instruction Interaction and Dependency

Multi-IF: Benchmarking LLMs on Multi-Turn and Multilingual Instructions Following

The Construction of Instruction-tuned LLMs for Finance without Instruction Data Using Continual Pretraining and Model Merging