Abstract:Personalized Federated Learning (PFL) is proposed to find the greatest personalized models for each client. To avoid the central failure and communication bottleneck in the server-based FL, we concentrate on the Decentralized Personalized Federated Learning (DPFL) that performs distributed model training in a Peer-to-Peer (P2P) manner. Most personalized works in DPFL are based on undirected and symmetric topologies, however, the data, computation and communication resources heterogeneity result in large variances in the personalized models, which lead the undirected aggregation to suboptimal personalized performance and unguaranteed convergence. To address these issues, we propose a directed collaboration DPFL framework by incorporating stochastic gradient push and partial model personalized, called \textbf{D}ecentralized \textbf{Fed}erated \textbf{P}artial \textbf{G}radient \textbf{P}ush (\textbf{DFedPGP}). It personalizes the linear classifier in the modern deep model to customize the local solution and learns a consensus representation in a fully decentralized manner. Clients only share gradients with a subset of neighbors based on the directed and asymmetric topologies, which guarantees flexible choices for resource efficiency and better convergence. Theoretically, we show that the proposed DFedPGP achieves a superior convergence rate of $\mathcal{O}(\frac{1}{\sqrt{T}})$ in the general non-convex setting, and prove the tighter connectivity among clients will speed up the convergence. The proposed method achieves state-of-the-art (SOTA) accuracy in both data and computation heterogeneity scenarios, demonstrating the efficiency of the directed collaboration and partial gradient push.

UA-PDFL: A Personalized Approach for Decentralized Federated Learning

Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Training

Group Personalized Federated Learning

Personalizing or Not: Dynamically Personalized Federated Learning with Incentives

DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training

A Unified Personalized Federated Learning Framework Ensuring Domain Generalization

Unifying Distillation with Personalization in Federated Learning

Decentralized Directed Collaboration for Personalized Federated Learning

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

Exploring Personalization Via Federated Representation Learning on Non-Iid Data.

Personalized and privacy-enhanced federated learning framework via knowledge distillation

PPFL: A Personalized Federated Learning Framework for Heterogeneous Population

Optimizing Personalized Federated Learning through Adaptive Layer-Wise Learning

A Personalized Federated Learning Framework Using Side Information for Heterogeneous Data Classification.

Feed: Towards Personalization-Effective Federated Learning

PLDP-FL: Federated Learning with Personalized Local Differential Privacy

Personalized Federated Learning with Hidden Information on Personalized Prior

Learn What You Need in Personalized Federated Learning

CD^2-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learning

Partially Personalized Federated Learning: Breaking the Curse of Data Heterogeneity

PPFed: A Privacy-Preserving and Personalized Federated Learning Framework