Abstract:Federated learning (FL) is a distributed machine learning paradigm under privacy preservation. However, data heterogeneity among clients leads to the shared global model obtained after training, which cannot fit the distribution of each client's dataset, and the performance of the model degrades. To address this problem, we proposed a personalized federated learning method based on clustering and knowledge distillation, called pFedCK. In this algorithm, each client has an interactive model that participates in global training and a personalized model that is only trained locally. Both of the models perform knowledge distillation with each other through the feature representation of the middle layer and the soft prediction of the model. In addition, in order to make an interaction model only obtaining the model information from the client, which has similar data distribution and avoids the interference of other heterogeneous information, the server will cluster the clients according to the similarity of the amount of parameter variation uploaded by different interaction models during every training round. By clustering clients, interaction models with similar data distributions can cooperate with each other to better fit the local dataset distribution. Thereby, the performance of personalized model can be improved by obtaining more valuable information indirectly. Finally, we conduct simulation experiments on three benchmark datasets under different data heterogeneity scenarios. Compared to the single model algorithms, the accuracy of pFedCK improved by an average of 23.4% and 23.8% over FedAvg and FedProx, respectively; compared to typical personalization algorithms, the accuracy of pFedCK improved by an average of 0.8% and 1.3%, and a maximum of 1.0% and 2.9% over FedDistill and FML.

Efficient One-off Clustering for Personalized Federated Learning

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

FedCE: Personalized Federated Learning Method based on Clustering Ensembles

FedSC: A federated learning algorithm based on client-side clustering

FedCOA: an Efficient Cluster Optimization for Federated Learning

FedSPD: A Soft-clustering Approach for Personalized Decentralized Federated Learning

A Personalized Federated Learning Method Based on Clustering and Knowledge Distillation

Federated Two Stage Decoupling With Adaptive Personalization Layers

FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering

A Multi-Task Based Clustering Personalized Federated Learning Method

Federated two-stage decoupling with adaptive personalization layers

An Efficient Client Clustering Algorithm for Clustered Federated Learning

FedFC: an Efficient Personalized Federated Learning Method on Non-iid Data

Personalized Federated Learning with Multiple Known Clusters

Reinforcement Federated Learning Method Based on Adaptive OPTICS Clustering

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

Adaptive Clustered Federated Learning with Representation Similarity

FedCLS:A Federated Learning Client Selection Algorithm Based on Cluster Label Information

A Personalized Federated Cloud-Edge Collaboration Framework Via Cross-Client Knowledge Distillation

A Clustering Federated Learning Algorithm Based on Three-Layer Structure

FedClust: Optimizing Federated Learning on Non-IID Data through Weight-Driven Client Clustering