Abstract:Personalized Federated Learning (PFL) is a commonly used framework that allows clients to collaboratively train their personalized models. PFL is particularly useful for handling situations where data from different clients are not independent and identically distributed (non-IID). Previous research in PFL implicitly assumes that clients can gain more benefits from those with similar data distributions. Correspondingly, methods such as personalized weight aggregation are developed to assign higher weights to similar clients during training. We pose a question: can a client benefit from other clients with dissimilar data distributions and if so, how? This question is particularly relevant in scenarios with a high degree of non-IID, where clients have widely different data distributions, and learning from only similar clients will lose knowledge from many other clients. We note that when dealing with clients with similar data distributions, methods such as personalized weight aggregation tend to enforce their models to be close in the parameter space. It is reasonable to conjecture that a client can benefit from dissimilar clients if we allow their models to depart from each other. Based on this idea, we propose DiversiFed which allows each client to learn from clients with diversified data distribution in personalized federated learning. DiversiFed pushes personalized models of clients with dissimilar data distributions apart in the parameter space while pulling together those with similar distributions. In addition, to achieve the above effect without using prior knowledge of data distribution, we design a loss function that leverages the model similarity to determine the degree of attraction and repulsion between any two models. Experiments on several datasets show that DiversiFed can benefit from dissimilar clients and thus outperform the state-of-the-art methods.

Pfedbea: Combatting Data Heterogeneity for Personalized Federated Learning by Body Exchange and Aggregation Abandon

FedPSE: Personalized Sparsification with Element-wise Aggregation for Federated Learning

FedPer++: Toward Improved Personalized Federated Learning on Heterogeneous and Imbalanced Data

FedDWA: Personalized Federated Learning with Dynamic Weight Adjustment

Pfedsim: Similarity-Aware Model Aggregation Towards Personalized Federated Learning.

PFED-AGG: A Personalized Private Federated Learning Aggregation Algorithm.

Improving Global Generalization and Local Personalization for Federated Learning

FedPA: An adaptively partial model aggregation strategy in Federated Learning

pFedES: Model Heterogeneous Personalized Federated Learning with Feature Extractor Sharing

FedFTHA: A Fine-Tuning and Head Aggregation Method in Federated Learning

Learn What You Need in Personalized Federated Learning

Personalized Federated Learning with Local Optimization Models

Towards Personalized Federated Learning via Heterogeneous Model Reassembly

The Diversity Bonus: Learning from Dissimilar Distributed Clients in Personalized Federated Learning

Personalized Federated Learning with Adaptive Feature Aggregation and Knowledge Transfer

FedReMa: Improving Personalized Federated Learning via Leveraging the Most Relevant Clients

Feed: Towards Personalization-Effective Federated Learning

FedAS: Bridging Inconsistency in Personalized Federated Learning

FedFC: an Efficient Personalized Federated Learning Method on Non-iid Data

An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

Adaptive Personalized Federated Learning for Heterogeneous Data: a Method Based on Parameter Decomposition and Continual Learning