Abstract:Large Language Models (LLMs), built on Transformer architectures, exhibit remarkable generalization across a wide range of tasks. However, fine-tuning these models for specific tasks remains resource-intensive due to their extensive parameterization. In this paper, we investigate two remarkable phenomena observed during the fine-tuning of LLMs, particularly focusing on the attention mechanism: (1) Different Impact, optimizing the $\mathbf{W}_v$ matrix significantly improves performance over optimizing the $\mathbf{W}_k$ matrix. Fine-tuning only the $\mathbf{W}_q$ and $\mathbf{W}_v$ matrices is computationally efficient, delivering results that are comparable to, or even better than, fine-tuning all three matrices $\mathbf{W}_q$, $\mathbf{W}_k$, and $\mathbf{W}_v$. (2) Efficient Convergence, employing distinct learning rates for these matrices is crucial for optimal performance, with a higher learning rate for the $\mathbf{W}_v$ matrix expediting convergence. However, theoretical analyses of these phenomena are still relatively limited. We present a theoretical analysis of these phenomena from two perspectives: (i) Generalization, where we demonstrate that fine-tuning only $\mathbf{W}_q$ and $\mathbf{W}_v$ improves generalization bounds, enhances memory efficiency, and (ii) Optimization, where we emphasize that the feature learning of the attention mechanism is efficient, particularly when using distinct learning rates for the matrices, which leads to more effective fine-tuning. Building on these insights, we propose a new strategy that improves fine-tuning efficiency in terms of both storage and time. Experimental results on benchmark datasets validate the effectiveness of this approach, supporting our theoretical findings. Our analysis lays the theoretical groundwork for configuring and improving lightweight algorithms in LLMs fine-tuning.

Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurts

FedAWS: A Federated Tuning Approach with Adaptive Weight Shrinking for Pre-trained Foundation Models

Fine-tuning can cripple your foundation model; preserving features may be the solution

LMFlow: An Extensible Toolkit for Finetuning and Inference of Large Foundation Models

HFT: Half Fine-Tuning for Large Language Models

Fine-Tuning is Fine, if Calibrated

Sheared Backpropagation for Fine-Tuning Foundation Models

Model Balancing Helps Low-data Training and Fine-tuning

FedPFT: Federated Proxy Fine-Tuning of Foundation Models

Balancing Speciality and Versatility: a Coarse to Fine Framework for Supervised Fine-tuning Large Language Model

Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning

In-Context Fine-Tuning for Time-Series Foundation Models

Enhancing Data Quality in Federated Fine-Tuning of Foundation Models

Text-guided Foundation Model Adaptation for Long-Tailed Medical Image Classification

Exploring Parameter-Efficient Fine-Tuning to Enable Foundation Models in Federated Learning

Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization

FeTT: Continual Class Incremental Learning via Feature Transformation Tuning

Partial Fine-Tuning: A Successor to Full Fine-Tuning for Vision Transformers

Preference-Oriented Supervised Fine-Tuning: Favoring Target Model Over Aligned Large Language Models

Learn From Model Beyond Fine-Tuning: A Survey

See Further for Parameter Efficient Fine-tuning by Standing on the Shoulders of Decomposition