Abstract:Personalized Federated Learning (PFL) is proposed to find the greatest personalized models for each client. To avoid the central failure and communication bottleneck in the server-based FL, we concentrate on the Decentralized Personalized Federated Learning (DPFL) that performs distributed model training in a Peer-to-Peer (P2P) manner. Most personalized works in DPFL are based on undirected and symmetric topologies, however, the data, computation and communication resources heterogeneity result in large variances in the personalized models, which lead the undirected aggregation to suboptimal personalized performance and unguaranteed convergence. To address these issues, we propose a directed collaboration DPFL framework by incorporating stochastic gradient push and partial model personalized, called \textbf{D}ecentralized \textbf{Fed}erated \textbf{P}artial \textbf{G}radient \textbf{P}ush (\textbf{DFedPGP}). It personalizes the linear classifier in the modern deep model to customize the local solution and learns a consensus representation in a fully decentralized manner. Clients only share gradients with a subset of neighbors based on the directed and asymmetric topologies, which guarantees flexible choices for resource efficiency and better convergence. Theoretically, we show that the proposed DFedPGP achieves a superior convergence rate of $\mathcal{O}(\frac{1}{\sqrt{T}})$ in the general non-convex setting, and prove the tighter connectivity among clients will speed up the convergence. The proposed method achieves state-of-the-art (SOTA) accuracy in both data and computation heterogeneity scenarios, demonstrating the efficiency of the directed collaboration and partial gradient push.

Byzantine-Robust and Communication-Efficient Personalized Federated Learning

Personalized Federated Learning towards Communication Efficiency, Robustness and Fairness

Byzantine-robust Variance-Reduced Federated Learning over Distributed Non-I.i.d. Data

DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training

Pfedlhns: Personalized Federated Learning Via Local Hypernetworks

Decentralized Personalized Online Federated Learning

Robust Federated Learning with Noisy Communication.

FedFC: an Efficient Personalized Federated Learning Method on Non-iid Data

Personalized Federated Learning with Local Optimization Models

Privacy-Preserving and Reliable Decentralized Federated Learning

Reliable and Interpretable Personalized Federated Learning

Sparse Federated Learning with Hierarchical Personalization Models.

Sparse Federated Learning with Hierarchical Personalized Models

Explicit Personalization and Local Training: Double Communication Acceleration in Federated Learning

Decentralized Directed Collaboration for Personalized Federated Learning

Communication-Efficient and Byzantine-Robust Distributed Stochastic Learning with Arbitrary Number of Corrupted Workers

Tensor Decomposition Based Personalized Federated Learning

A Decentralized Federated Learning Framework Via Committee Mechanism with Convergence Guarantee

Personalized Federated Learning: an Attentive Collaboration Approach

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

Learn What You Need in Personalized Federated Learning