Abstract:We study the problem of fine-tuning a language model (LM) for a target task by optimally using the information from $n$ auxiliary tasks. This problem has broad applications in NLP, such as targeted instruction tuning and data selection in chain-of-thought fine-tuning. The key challenge of this problem is that not all auxiliary tasks are useful to improve the performance of the target task. Thus, choosing the right subset of auxiliary tasks is crucial. Conventional subset selection methods, such as forward and backward stepwise selection, are unsuitable for LM fine-tuning because they require repeated training on subsets of auxiliary tasks. This paper introduces a new algorithm to estimate model fine-tuning performances without repeated training. Our algorithm first performs multitask training using the data of all the tasks to obtain a meta initialization. Then, we approximate the model fine-tuning loss of a subset using functional values and gradients from the meta initialization. Empirically, we find that this gradient-based approximation holds with remarkable accuracy for twelve transformer-based LMs. Thus, we can now estimate fine-tuning performances on CPUs within a few seconds. Finally, we fine-tune the pretrained base model for once on the selected subset of tasks. We conduct extensive experiments to validate this approach, delivering a speedup of $30\times$ over conventional subset selection while incurring only $1\%$ error of the true fine-tuning performances. In downstream evaluations involving both instruction tuning and chain-of-thought fine-tuning, this loss-based selection approach improves over prior gradient or representation similarity-based methods for subset selection by up to $3.8\%$.

Composable Sparse Fine-Tuning for Cross-Lingual Transfer

Sparse is Enough in Fine-tuning Pre-trained Large Language Models

Empirical Analysis of Efficient Fine-Tuning Methods for Large Pre-Trained Language Models

Refining Salience-Aware Sparse Fine-Tuning Strategies for Language Models

Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approach

Adapting Language Models via Token Translation

Parameter-Efficient Fine-Tuning With Adapters

VL-Adapter: Parameter-Efficient Transfer Learning for Vision-and-Language Tasks

Parameter-Efficient Transfer Learning for NLP

Sparse-Tuning: Adapting Vision Transformers with Efficient Fine-tuning and Inference

SPAFIT: Stratified Progressive Adaptation Fine-tuning for Pre-trained Large Language Models

SparseAdapter: An Easy Approach for Improving the Parameter-Efficiency of Adapters

Gradient Sparsification For Masked Fine-Tuning of Transformers

SQFT: Low-cost Model Adaptation in Low-precision Sparse Foundation Models

ScaLearn: Simple and Highly Parameter-Efficient Task Transfer by Learning to Scale

Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning

Code-switching finetuning: Bridging multilingual pretrained language models for enhanced cross-lingual performance

Skip Tuning: Pre-trained Vision-Language Models are Effective and Efficient Adapters Themselves

Multilingual Speech Translation with Efficient Finetuning of Pretrained Models

Analyzing and Reducing the Performance Gap in Cross-Lingual Transfer with Fine-tuning Slow and Fast

Parameter-efficient Tuning for Large Language Model Without Calculating Its Gradients