Abstract:Federated learning (FL) is an emerging distributed machine learning paradigm that enables collaborative training of machine learning models over decentralized devices without exposing their local data. One of the major challenges in FL is the presence of uneven data distributions across client devices, violating the well-known assumption of independent-and-identically-distributed (IID) training samples in conventional machine learning. To address the performance degradation issue incurred by such data heterogeneity, clustered federated learning (CFL) shows its promise by grouping clients into separate learning clusters based on the similarity of their local data distributions. However, state-of-the-art CFL approaches require a large number of communication rounds to learn the distribution similarities during training until the formation of clusters is stabilized. Moreover, some of these algorithms heavily rely on a predefined number of clusters, thus limiting their flexibility and adaptability. In this paper, we propose {\em FedClust}, a novel approach for CFL that leverages the correlation between local model weights and the data distribution of clients. {\em FedClust} groups clients into clusters in a one-shot manner by measuring the similarity degrees among clients based on the strategically selected partial weights of locally trained models. We conduct extensive experiments on four benchmark datasets with different non-IID data settings. Experimental results demonstrate that {\em FedClust} achieves higher model accuracy up to $\sim$45\% as well as faster convergence with a significantly reduced communication cost up to 2.7$\times$ compared to its state-of-the-art counterparts.

Towards Hierarchical Clustered Federated Learning with Model Stability on Mobile Devices

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

Hierarchical Federated Learning: the Interplay of User Mobility and Data Heterogeneity

Communication-efficient clustered federated learning via model distance

Flexible Clustered Federated Learning for Client-Level Data Distribution Shift

Hierarchical federated learning based on wireless D2D networks

Stochastic Clustered Federated Learning

A dynamic adaptive iterative clustered federated learning scheme

LayerCFL: an Efficient Federated Learning with Layer-Wised Clustering

Adaptive Client Clustering for Efficient Federated Learning over Non-IID and Imbalanced Data

Information-Exchangeable Hierarchical Clustering for Federated Learning with Non-IID Data

FedClust: Optimizing Federated Learning on Non-IID Data through Weight-Driven Client Clustering

FedAC: An Adaptive Clustered Federated Learning Framework for Heterogeneous Data

FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering

Clustered Federated Learning in Heterogeneous Environment.

Adaptive Clustering-Based Model Aggregation for Federated Learning with Imbalanced Data

Clustered Federated Learning with Weighted Model Aggregation for Imbalanced Data

Comparative Evaluation of Clustered Federated Learning Methods

Fuzzy Clustered Federated Learning under Mixed Data Distributions.

A Comprehensive Study on Personalized Federated Learning with Non-IID Data.

Clustered Federated Learning with Model Integration for Non-IID Data in Wireless Networks.