Abstract:One of the main challenges of federated learning (FL) is handling non-independent and identically distributed (non-IID) client data, which may occur in practice due to unbalanced datasets and use of different data sources across clients. Knowledge sharing and model personalization are key strategies for addressing this issue. Clustered federated learning is a class of FL methods that groups clients that observe similarly distributed data into clusters, such that every client is typically associated with one data distribution and participates in training a model for that distribution along their cluster peers. In this paper, we present a unified Bayesian framework for clustered FL which associates clients to clusters. Then we propose several practical algorithms to handle the, otherwise growing, data associations in a way that trades off performance and computational complexity. This work provides insights on client-cluster associations and enables client knowledge sharing in new ways. The proposed framework circumvents the need for unique client-cluster associations, which is seen to increase the performance of the resulting models in a variety of experiments.

What problem does this paper attempt to address?

### What problems does this paper attempt to solve? This paper aims to solve an important challenge in Federated Learning (FL), that is, how to handle non - independent and identically distributed (non - IID) client data. Specifically, the paper focuses on the following two main problems: 1. **Clients are clustered into non - overlapping clusters**: - In current Clustered Federated Learning (CFL) methods, clients are usually clustered into non - overlapping clusters. This means that each client can only belong to one cluster, and after several rounds of training, the participating clients tend to converge to the same cluster quickly. This may lead to inefficient use of local information and affect model performance. 2. **Lack of a unified theory to describe client - cluster association and knowledge sharing**: - Existing CFL methods lack a unified theoretical framework to describe information sharing among clients and their contributions to multiple model trainings when dealing with non - IID data. In particular, these methods usually rely on heuristic selection strategies for client - cluster association without theoretically optimizing this association. ### Solutions proposed in the paper To solve the above problems, the paper proposes a new Bayesian Framework, reinterpreting CFL as a Bayesian data association problem. Specific contributions include: 1. **A novel Bayesian framework**: - Model CFL as a Bayesian data association problem, draw on methods in fields such as multi - target tracking, and propose a new theoretical framework to handle client - cluster association. 2. **Efficient hypothesis management**: - Propose three algorithms (BCFL - G, BCFL - C and BCFL - MH) to manage the association hypotheses that grow rapidly with the increase of communication rounds, balancing computational efficiency and performance. 3. **Superior performance on non - IID data**: - Experimental results show that the proposed method outperforms existing CFL algorithms in non - IID data settings with feature - skew and label - skew. 4. **New research directions**: - By redefining personalized and clustered FL as a client - cluster association problem, provide a new paradigm that may inspire further algorithmic innovation in the field of Federated Learning. ### Summary By introducing the Bayesian framework, the paper not only solves the limitations of existing CFL methods in dealing with non - IID data, but also provides a unified theoretical framework to optimize client - cluster association and knowledge sharing, thereby improving model performance.

A Bayesian Framework for Clustered Federated Learning

A Bayesian Federated Learning Framework With Online Laplace Approximation

Federated Learning via Variational Bayesian Inference: Personalization, Sparsity and Clustering

On the Convergence of Clustered Federated Learning

A Clustered Federated Learning Method of User Behavior Analysis Based on Non-IID Data

Stochastic Clustered Federated Learning

Dynamic Clustering Federated Learning for Non-IID Data.

Clustered FedStack: Intermediate Global Models with Bayesian Information Criterion

Towards Client Driven Federated Learning

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

FedClust: Optimizing Federated Learning on Non-IID Data through Weight-Driven Client Clustering

Clustered federated learning based on nonconvex pairwise fusion

An Efficient Framework for Clustered Federated Learning

Federated Bayesian Deep Learning: The Application of Statistical Aggregation Methods to Bayesian Models

FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering

Bayesian Federated Learning: A Survey

Clustered Data Sharing for Non-IID Federated Learning over Wireless Networks

Federated PAC-Bayesian Learning on Non-IID data

A Personalized Federated Learning Method Based on Clustering and Knowledge Distillation