Abstract:As a popular paradigm of distributed learning, personalized federated learning (PFL) allows personalized models to improve generalization ability and robustness by utilizing knowledge from all distributed clients. Most existing PFL algorithms tackle personalization in a model-centric way, such as personalized layer partition, model regularization, and model interpolation, which all fail to take into account the data characteristics of distributed clients. In this paper, we propose a novel PFL framework for image classification tasks, dubbed pFedPT, that leverages personalized visual prompts to implicitly represent local data distribution information of clients and provides that information to the aggregation model to help with classification tasks. Specifically, in each round of pFedPT training, each client generates a local personalized prompt related to local data distribution. Then, the local model is trained on the input composed of raw data and a visual prompt to learn the distribution information contained in the prompt. During model testing, the aggregated model obtains prior knowledge of the data distributions based on the prompts, which can be seen as an adaptive fine-tuning of the aggregation model to improve model performances on different clients. Furthermore, the visual prompt can be added as an orthogonal method to implement personalization on the client for existing FL methods to boost their performance. Experiments on the CIFAR10 and CIFAR100 datasets show that pFedPT outperforms several state-of-the-art (SOTA) PFL algorithms by a large margin in various settings.

Personalized Interpretation on Federated Learning: A Virtual Concepts approach

FedDGP: Disentangling Global and Personal Models for Federated Learning

FediOS: Decoupling Orthogonal Subspaces for Personalization in Feature-skew Federated Learning

Personalization Disentanglement for Federated Learning: An explainable perspective

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Personalizing or Not: Dynamically Personalized Federated Learning with Incentives

Group Personalized Federated Learning

Personalized Federated Learning with Multi-View Geometry Structure

Exploiting Personalized Invariance for Better Out-of-distribution Generalization in Federated Learning

Personalized Federated Learning towards Communication Efficiency, Robustness and Fairness

Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity

IP-FL: Incentivized and Personalized Federated Learning

Personalized Federated Learning with First Order Model Optimization

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning

Sparse Personalized Federated Learning

A Survey of Federated Learning on Non-IID Data

Personalized federated learning based on feature fusion

Client-supervised Federated Learning: Towards One-model-for-all Personalization

Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clustering

Visual Prompt Based Personalized Federated Learning