Abstract:Federated learning (FL) aims to raise a multi-client collaboration approach in the case of local data isolation. In particular, the clients with non-IID data frequently participate in or leave the federated learning training process asynchronously, resulting in dynamic federated learning (DFL) scenario, which attracts more and more attention. Indeed, an effective DFL solution has to address the following two challenges: 1) Statistical Dynamics. The distributions of local data from clients are always non-IID and the global data distribution is dynamic due to the participation or departure of clients. 2) Expiration Dynamics. After clients leave the federated training process, their historical updated models have a certain validity to reuse in subsequent rounds but it is hard to quantify this validity. In this paper, we first consider clustering the clients with similar data distribution to make them much closer to IID and concentrating on the training the models in each cluster. Then we analyze the changing trend of model validity named model quality and define one suitable function to describe expiration dynamics. As a solution, we propose Dynamic Clustering Federated Learning (DCFL) framework to improve federated learning on non-IID data in DFL. Specifically, DCFL follows the client-server architecture as the standard FL. On the client side, the local devices calculate the related information of the local data distribution for client clustering. On the server side, we design two strategies for the challenges above. We propose dynamic clustering aggregation strategy (including a dynamic clustering algorithm and a two-stage aggregation) by dynamically clustering clients and then aggregating the local models to overcome Statistical Dynamics. Besides, we propose expiration memory strategy by reusing the historical models and then adjusting to model quality of historical models as the basis for model aggregation to overcome Expiration Dynamics. Finally, we conduct extensive experiments on public datasets, which demonstrate the effectiveness of the DCFL framework.

CDFed: Contribution-based Dynamic Federated Learning for Managing System and Statistical Heterogeneity

Client Contribution Normalization for Enhanced Federated Learning

An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

Towards Client Driven Federated Learning

Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selection

Flexible Clustered Federated Learning for Client-Level Data Distribution Shift

FedClust: Tackling Data Heterogeneity in Federated Learning through Weight-Driven Client Clustering

Contribution‐based Federated Learning client selection

FedEP: Tailoring Attention to Heterogeneous Data Distribution with Entropy Pooling for Decentralized Federated Learning

FedDUAL: A Dual-Strategy with Adaptive Loss and Dynamic Aggregation for Mitigating Data Heterogeneity in Federated Learning

Dynamic Clustering Federated Learning for Non-IID Data.

FedECADO: A Dynamical System Model of Federated Learning

Federated Learning with Dynamic Client Arrival and Departure: Convergence and Rapid Adaptation via Initial Model Construction

FedPD: A Federated Learning Framework With Adaptivity to Non-IID Data

FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity

Federated Learning under Heterogeneous and Correlated Client Availability

Federated learning with workload-aware client scheduling in heterogeneous systems

Accelerating Federated Learning via Sequential Training of Grouped Heterogeneous Clients

FED-SDS: Adaptive Structured Dynamic Sparsity for Federated Learning under Heterogeneous Clients

Communication-Efficient Federated Learning With Data and Client Heterogeneity

FedDC: Federated Learning with Non-IID Data via Local Drift Decoupling and Correction