Abstract:Federated Domain Adaptation (FDA) is a Federated Learning (FL) scenario where models are trained across multiple clients with unique data domains but a shared category space, without transmitting private data. The primary challenge in FDA is data heterogeneity, which causes significant divergences in gradient updates when using conventional averaging-based aggregation methods, reducing the efficacy of the global model. This further undermines both in-domain and out-of-domain performance (within the same federated system but outside the local client). To address this, we propose a novel framework called \textbf{M}ulti-domain \textbf{P}rototype-based \textbf{F}ederated Fine-\textbf{T}uning (MPFT). MPFT fine-tunes a pre-trained model using multi-domain prototypes, i.e., pretrained representations enriched with domain-specific information from category-specific local data. This enables supervised learning on the server to derive a globally optimized adapter that is subsequently distributed to local clients, without the intrusion of data privacy. Empirical results show that MPFT significantly improves both in-domain and out-of-domain accuracy over conventional methods, enhancing knowledge preservation and adaptation in FDA. Notably, MPFT achieves convergence within a single communication round, greatly reducing computation and communication costs. To ensure privacy, MPFT applies differential privacy to protect the prototypes. Additionally, we develop a prototype-based feature space hijacking attack to evaluate robustness, confirming that raw data samples remain unrecoverable even after extensive training epochs. The complete implementation of MPFL is available at \url{<a class="link-external link-https" href="https://anonymous.4open.science/r/DomainFL/" rel="external noopener nofollow">this https URL</a>}.

FEDBFPT: an Efficient Federated Learning Framework for BERT Further Pre-Training

FedDGP: Disentangling Global and Personal Models for Federated Learning

Federated Split BERT for Heterogeneous Text Classification

Conquering the Communication Constraints to Enable Large Pre-Trained Models in Federated Learning

FedPT: Federated Proxy-Tuning of Large Language Models on Resource-Constrained Edge Devices

Federated Foundation Models: Privacy-Preserving and Collaborative Learning for Large Models

When Federated Learning Meets Pre-trained Language Models' Parameter-Efficient Tuning Methods

FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models

Federated Learning from Pre-Trained Models: A Contrastive Learning Approach

FedPFT: Federated Proxy Fine-Tuning of Foundation Models

Communication-Efficient and Tensorized Federated Fine-Tuning of Large Language Models

Fed-urlBERT: Client-side Lightweight Federated Transformers for URL Threat Analysis

A Survey on Efficient Federated Learning Methods for Foundation Model Training

Scaling Federated Learning for Fine-tuning of Large Language Models

FedDBL: Communication and Data Efficient Federated Deep-Broad Learning for Histopathological Tissue Classification

Enhancing Federated Learning by One-Shot Transferring of Intermediate Features from Clients

BAFFLE: A Baseline of Backpropagation-Free Federated Learning

Enhancing Federated Domain Adaptation with Multi-Domain Prototype-Based Federated Fine-Tuning

PPFed: A Privacy-Preserving and Personalized Federated Learning Framework

Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learning