Abstract:A large amount of instructional text data is essential to enhance the performance of pre-trained large language models (LLMs) for downstream tasks. This data can contain sensitive information and therefore cannot be shared in practice, resulting in data silos that limit the effectiveness of LLMs on various tasks. Federated learning (FL) enables collaborative fine-tuning across different clients without sharing their data. Nonetheless, in practice, this instructional text data is highly heterogeneous in both quantity and distribution across clients, necessitating distinct model structures to best accommodate the variations. However, existing federated fine-tuning approaches either enforce the same model structure or rely on predefined ad-hoc architectures unaware of data distribution, resulting in suboptimal performance. To address this challenge, we propose FedAMoLE, a lightweight personalized federated fine-tuning framework that leverages data-driven heterogeneous model architectures. FedAMoLE introduces the Adaptive Mixture of LoRA Experts (AMoLE) module, which facilitates model heterogeneity with minimal communication overhead by allocating varying numbers of LoRA-based domain experts to each client. Furthermore, we develop a reverse selection-based expert assignment (RSEA) strategy, which enables data-driven model architecture adjustment during fine-tuning by allowing domain experts to select clients that best align with their knowledge domains. Extensive experiments across six different scenarios of data heterogeneity demonstrate that FedAMoLE significantly outperforms existing methods for federated LLM fine-tuning, achieving superior accuracy while maintaining good scalability.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are: 1. **Data silo problem**: Large - scale language models (LLMs) require a large amount of instructional text data to improve performance when performing downstream tasks. However, this data may contain sensitive information and thus cannot be directly shared, resulting in the phenomenon of data silos, which limits the effectiveness of LLMs on various tasks. 2. **Problem of personalized model adaptability under heterogeneous data distribution**: In practical applications, the data of different clients is highly heterogeneous in quantity and distribution. Existing federated fine - tuning methods either force the use of the same model structure or rely on predefined architectures that are not sensitive to data distribution, which leads to sub - optimal performance. 3. **Limitations of existing methods**: - **Limited support for model heterogeneity**: In highly heterogeneous scenarios, the amount of data of some clients is very limited, and different - sized models need to be used to avoid over - fitting. However, existing federated fine - tuning methods are often difficult to effectively support model heterogeneity. - **Data - unaware model structure**: Mainstream federated fine - tuning methods mainly rely on manually predefined model architectures. These architectures usually only consider local resource constraints or local data characteristics and fail to comprehensively consider cross - client data features, resulting in poor performance in statistically heterogeneous data environments. To solve these problems, the paper proposes FedAMoLE, a lightweight personalized federated fine - tuning framework. By introducing the Adaptive Mixture of LoRA Experts (AMoLE) module and supporting the data - driven model architecture adjustment strategy (RSEA), it realizes model heterogeneity and data - aware model architecture design with low communication overhead. Specifically, FedAMoLE aims to achieve the following goals: - Support model heterogeneity at the architecture level to better adapt to the data distribution of different clients. - Customize a personalized model architecture for each client by jointly considering local and global data features. - Achieve the above goals with a reasonable additional communication overhead. Through these improvements, FedAMoLE significantly improves the federated LLM fine - tuning performance in heterogeneous data environments while maintaining good scalability.

Personalized Federated Fine-Tuning for LLMs via Data-Driven Heterogeneous Model Architectures

FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts

FedDGP: Disentangling Global and Personal Models for Federated Learning

FLoRA: Federated Fine-Tuning Large Language Models with Heterogeneous Low-Rank Adaptations

Federated Mutual Learning: a Collaborative Machine Learning Method for Heterogeneous Data, Models, and Objectives

FedAPEN: Personalized Cross-silo Federated Learning with Adaptability to Statistical Heterogeneity

Federated LLMs Fine-tuned with Adaptive Importance-Aware LoRA

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Federated Fine-tuning of Large Language Models under Heterogeneous Tasks and Client Resources

FedBiOT: LLM Local Fine-tuning in Federated Learning without Full Model

pFedMoE: Data-Level Personalization with Mixture of Experts for Model-Heterogeneous Personalized Federated Learning

Federated Large Language Model: Solutions, Challenges and Future Directions

Fisher Information-based Efficient Curriculum Federated Learning with Large Language Models

Towards Efficient Model-Heterogeneity Federated Learning for Large Models

FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clients

FedMLLM: Federated Fine-tuning MLLM on Multimodal Heterogeneity Data

FederatedScope-LLM: A Comprehensive Package for Fine-tuning Large Language Models in Federated Learning

FedLoRA: Model-Heterogeneous Personalized Federated Learning with LoRA Tuning

pFedAFM: Adaptive Feature Mixture for Batch-Level Personalization in Heterogeneous Federated Learning

Towards Personalized Federated Learning via Heterogeneous Model Reassembly

Adaptive Personalized Federated Learning for Heterogeneous Data: a Method Based on Parameter Decomposition and Continual Learning