Abstract: Fine-tuning a pre-trained model (such as BERT, ALBERT, RoBERTa, T5, GPT, etc.) has proven to be one of the most promising paradigms in recent NLP research. However, numerous recent works indicate that fine-tuning suffers from the instability problem, i.e., tuning the same model under the same setting results in significantly different performance. Many recent works have proposed different methods to solve this problem, but there is no theoretical understanding of why and how these methods work. In this paper, we propose a novel theoretical stability analysis of fine-tuning that focuses on two commonly used settings, namely, full fine-tuning and head tuning. We define the stability under each setting and prove the corresponding stability bounds. The theoretical bounds explain why and how several existing methods can stabilize the fine-tuning procedure. In addition to being able to explain most of the observed empirical discoveries, our proposed theoretical analysis framework can also help in the design of effective and provable methods. Based on our theory, we propose three novel strategies to stabilize the fine-tuning procedure, namely, Maximal Margin Regularizer (MMR), Multi-Head Loss (MHLoss), and Self Unsupervised Re-Training (SURT). We extensively evaluate our proposed approaches on 11 widely used real-world benchmark datasets, as well as hundreds of synthetic classification datasets. The experiment results show that our proposed methods significantly stabilize the fine-tuning procedure and also corroborate our theoretical analysis.

A fine-tuning approach research of pre-trained model with two stage

Can Fine-tuning Pre-trained Models Lead to Perfect NLP? A Study of the Generalizability of Relation Extraction.

How to Fine-Tune BERT for Text Classification?

A Closer Look at How Fine-tuning Changes BERT

Investigating Learning Dynamics of BERT Fine-Tuning

A Stability Analysis of Fine-Tuning a Pre-Trained Model

Improving BERT Fine-Tuning via Self-Ensemble and Self-Distillation

Towards Efficient Fine-tuning of Pre-trained Code Models: an Experimental Study and Beyond

KNN-BERT: Fine-Tuning Pre-Trained Models with KNN Classifier

Revisiting K-Nn for Fine-Tuning Pre-trained Language Models

Layer-wise Learning Rate Optimization for Task-Dependent Fine-Tuning of Pre-trained Models: An Evolutionary Approach

A Robustly Optimized BERT Pre-training Approach with Post-training

Single task fine-tune BERT for text classification

Empirical Analysis of Efficient Fine-Tuning Methods for Large Pre-Trained Language Models

Visualizing and Understanding the Effectiveness of BERT.

Fine-Tuning BERT for Sentiment Analysis of Vietnamese Reviews

Methods to Enhance BERT in Aspect-Based Sentiment Classification

Output Layer Go First: Better Fine-tuning by Bridging the Gap with Pre-training

Amuro & Char: Analyzing the Relationship between Pre-Training and Fine-Tuning of Large Language Models

HyPe: Better Pre-trained Language Model Fine-tuning with Hidden Representation Perturbation

LayerNorm: A key component in parameter-efficient fine-tuning