Abstract:Federated learning (FL) facilitates a privacy-preserving neural network training paradigm through collaboration between edge clients and a central server. One significant challenge is that the distributed data is not independently and identically distributed (non-IID), typically including both intra-domain and inter-domain heterogeneity. However, recent research is limited to simply using averaged signals as a form of regularization and only focusing on one aspect of these non-IID challenges. Given these limitations, this paper clarifies these two non-IID challenges and attempts to introduce cluster representation to address them from both local and global perspectives. Specifically, we propose a dual-clustered feature contrast-based FL framework with dual focuses. First, we employ clustering on the local representations of each client, aiming to capture intra-class information based on these local clusters at a high level of granularity. Then, we facilitate cross-client knowledge sharing by pulling the local representation closer to clusters shared by clients with similar semantics while pushing them away from clusters with dissimilar semantics. Second, since the sizes of local clusters belonging to the same class may differ for each client, we further utilize clustering on the global side and conduct averaging to create a consistent global signal for guiding each local training in a contrastive manner. Experimental results on multiple datasets demonstrate that our proposal achieves comparable or superior performance gain under intra-domain and inter-domain heterogeneity.

Dual-Segment Clustering Strategy for Hierarchical Federated Learning in Heterogeneous Wireless Environments

Dual-Segment Clustering Strategy for Federated Learning in Heterogeneous Environments.

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

Hierarchical Federated Learning: the Interplay of User Mobility and Data Heterogeneity

Hierarchical federated learning based on wireless D2D networks

Fuzzy Clustered Federated Learning under Mixed Data Distributions.

Joint Client Scheduling and Wireless Resource Allocation for Heterogeneous Federated Edge Learning with Non-IID Data

Clustered Federated Learning in Heterogeneous Environment.

FedCCL: Federated Dual-Clustered Feature Contrast Under Domain Heterogeneity

SoFL: Clustered Federated Learning Based on Dual Clustering for Heterogeneous Data

Clustered Data Sharing for Non-IID Federated Learning over Wireless Networks

FedHelo: Hierarchical Federated Learning with Loss-Based-Heterogeneity in Wireless Networks

FedDUAL: A Dual-Strategy with Adaptive Loss and Dynamic Aggregation for Mitigating Data Heterogeneity in Federated Learning

Information-Exchangeable Hierarchical Clustering for Federated Learning with Non-IID Data

Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity

Sequential Federated Learning in Hierarchical Architecture on Non-IID Datasets

Scalable Hierarchical Over-the-Air Federated Learning

Adaptive Hierarchical Federated Learning Over Wireless Networks

Federated learning with incremental clustering for heterogeneous data

Adaptive Clustering Based Straggler-aware Federated Learning in Wireless Edge Networks

Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing