Personalized Collaborative Fine-Tuning for On-Device Large Language Models

Nicolas Wagner,Dongyang Fan,Martin Jaggi

2024-08-07

Abstract:We explore on-device self-supervised collaborative fine-tuning of large language models with limited local data availability. Taking inspiration from the collaborative learning community, we introduce three distinct trust-weighted gradient aggregation schemes: weight similarity-based, prediction similarity-based and validation performance-based. To minimize communication overhead, we integrate Low-Rank Adaptation (LoRA) and only exchange LoRA weight updates. Our protocols, driven by prediction and performance metrics, surpass both FedAvg and local fine-tuning methods, which is particularly evident in realistic scenarios with more diverse local data distributions. The results underscore the effectiveness of our approach in addressing heterogeneity and scarcity within local datasets.

Computation and Language,Machine Learning

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the personalized collaborative fine - tuning of large language models (LLMs) on devices, especially in the case of limited local data. Specifically, the paper explores how to improve the personalized performance of large language models through user - to - user collaboration by leveraging limited local data resources while protecting user privacy. Since the local data distribution of each user may be heterogeneous and scarce, traditional local fine - tuning methods may not be effective. Therefore, new collaborative strategies need to be developed to overcome these challenges. The paper proposes three trust - weighted gradient aggregation schemes: weight - similarity - based, prediction - similarity - based, and verification - performance - based. These schemes aim to improve the effectiveness of collaborative fine - tuning by reducing communication overhead (such as only exchanging low - rank adaptation (LoRA) weight updates) and optimizing the gradient aggregation process. Experimental results show that the proposed collaborative protocol is superior to FedAvg and local fine - tuning methods in dealing with data heterogeneity and scarcity, especially when there is a more diverse local data distribution in real - world scenarios. In conclusion, the goal of this research is to explore the possibility of personalized collaborative fine - tuning on devices to overcome local data limitations and improve the personalized performance of large language models in different user environments.

Personalized Collaborative Fine-Tuning for On-Device Large Language Models

Federated Low-Rank Adaptation for Large Models Fine-Tuning over Wireless Networks

Enabling On-Device Large Language Model Personalization with Self-Supervised Data Selection and Synthesis

Federated LLMs Fine-tuned with Adaptive Importance-Aware LoRA

MIRA: A Method of Federated MultI-Task Learning for LaRge LAnguage Models

Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources

Towards Robust and Efficient Federated Low-Rank Adaptation with Heterogeneous Clients

PocketLLM: Enabling On-Device Fine-Tuning for Personalized LLMs

Federated Full-Parameter Tuning of Billion-Sized Language Models with Communication Cost under 18 Kilobytes

LoRA$^2$ : Multi-Scale Low-Rank Approximations for Fine-Tuning Large Language Models

Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities

LaMDA: Large Model Fine-Tuning via Spectrally Decomposed Low-Dimensional Adaptation

Adaptive Self-Supervised Learning Strategies for Dynamic On-Device LLM Personalization

Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning

Personalized Wireless Federated Learning for Large Language Models

Less is More: Extreme Gradient Boost Rank-1 Adaption for Efficient Finetuning of LLMs

AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning

On-Device Collaborative Language Modeling via a Mixture of Generalists and Specialists

CombLM: Adapting Black-Box Language Models through Small Fine-Tuned Models

Chain of LoRA: Efficient Fine-tuning of Language Models via Residual Learning

Asynchronous Local-SGD Training for Language Modeling