Abstract:Federated Learning (FL) aims to train machine learning models for multiple clients without sharing their own private data. Due to the heterogeneity of clients' local data distribution, recent studies explore the personalized FL that learns and deploys distinct local models with the help of auxiliary global models. However, the clients can be heterogeneous in terms of not only local data distribution, but also their computation and communication resources. The capacity and efficiency of personalized models are restricted by the lowest-resource clients, leading to sub-optimal performance and limited practicality of personalized FL. To overcome these challenges, we propose a novel approach named pFedGate for efficient personalized FL by adaptively and efficiently learning sparse local models. With a lightweight trainable gating layer, pFedGate enables clients to reach their full potential in model capacity by generating different sparse models accounting for both the heterogeneous data distributions and resource constraints. Meanwhile, the computation and communication efficiency are both improved thanks to the adaptability between the model sparsity and clients' resources. Further, we theoretically show that the proposed pFedGate has superior complexity with guaranteed convergence and generalization error. Extensive experiments show that pFedGate achieves superior global accuracy, individual accuracy and efficiency simultaneously over state-of-the-art methods. We also demonstrate that pFedGate performs better than competitors in the novel clients participation and partial clients participation scenarios, and can learn meaningful sparse local models adapted to different data distributions.

Robust Personalized Federated Learning with Sparse Penalization*

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

Sparse Personalized Federated Learning

Achieving Personalized Federated Learning with Sparse Local Models

FedAPEN: Personalized Cross-silo Federated Learning with Adaptability to Statistical Heterogeneity

Sparse Federated Learning with Hierarchical Personalized Models

Personalized Multi-tier Federated Learning

CosPer: An adaptive personalized approach for enhancing fairness and robustness of federated learning

Group Personalized Federated Learning

Learn What You Need in Personalized Federated Learning

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

A personalized federated learning method based on the residual multi-head attention mechanism

FedL2P: Federated Learning to Personalize

Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data

Personalized Federated Learning via Feature Distribution Adaptation

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

Personalizing or Not: Dynamically Personalized Federated Learning with Incentives

Efficient Personalized Federated Learning via Sparse Model-Adaptation

Sparse Training for Federated Learning with Regularized Error Correction

Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity

Optimizing Personalized Federated Learning through Adaptive Layer-Wise Learning