Abstract:Federated learning, as a promising distributed learning paradigm, enables collaborative training of a global model across multiple network edge clients without the need for central data collecting. However, the heterogeneity of edge data distribution drags the model towards the local minima, which can be distant from the global optimum. Such heterogeneity often leads to slow convergence and substantial communication overhead. To address these issues, we propose a novel federated learning framework called FedCMD, a model decoupling tailored to the Cloud-edge supported federated learning that separates deep neural networks into a body for capturing shared representations in Cloud and a personalized head for migrating data heterogeneity. Our motivation is that, by the deep investigation of the performance of selecting different neural network layers as the personalized head, we found rigidly assigning the last layer as the personalized head in current studies is not always optimal. Instead, it is necessary to dynamically select the personalized layer that maximizes the training performance by taking the representation difference between neighbor layers into account. To find the optimal personalized layer, we utilize the low-dimensional representation of each layer to contrast feature distribution transfer and introduce a Wasserstein-based layer selection method, aimed at identifying the best-match layer for personalization. Additionally, a weighted global aggregation algorithm is proposed based on the selected personalized layer for the practical application of FedCMD. Extensive experiments on ten benchmarks demonstrate the efficiency and superior performance of our solution compared with nine state-of-the-art solutions. All code and results are available at

MpFedcon : Model-Contrastive Personalized Federated Learning with the Class Center

Personalized federated learning based on feature fusion

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

Federated Learning with Label Distribution Skew via Logits Calibration.

DSFedCon: Dynamic Sparse Federated Contrastive Learning for Data-Driven Intelligent Systems

FedCRL: Personalized Federated Learning with Contrastive Shared Representations for Label Heterogeneity in Non-IID Data

PPFed: A Privacy-Preserving and Personalized Federated Learning Framework

Few-Shot Model Agnostic Federated Learning

Towards Optimal Customized Architecture for Heterogeneous Federated Learning with Contrastive Cloud-Edge Model Decoupling

Sparse Personalized Federated Learning

FedCL: Federated Contrastive Learning for Multi-center Medical Image Classification

Federated Learning Versus Classical Machine Learning: A Convergence Comparison

Federated Contrastive Learning for Personalized Semantic Communication

FedSSA: Semantic Similarity-based Aggregation for Efficient Model-Heterogeneous Personalized Federated Learning

Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data

Federated Learning with Personalization Layers

FedFed: Feature Distillation Against Data Heterogeneity in Federated Learning

FedSLD: Federated Learning with Shared Label Distribution for Medical Image Classification

Federated Learning via Consensus Mechanism on Heterogeneous Data: A New Perspective on Convergence

Completely Heterogeneous Federated Learning