Abstract:Federated learning (FL) is a collaborative machine learning framework that requires different clients (e.g., Internet of Things devices) to participate in the machine learning model training process by training and uploading their local models to an FL server in each global iteration. Upon receiving the local models from all the clients, the FL server generates a global model by aggregating the received local models. This traditional FL process may suffer from the straggler problem in heterogeneous client settings, where the FL server has to wait for slow clients to upload their local models in each global iteration, thus increasing the overall training time. One of the solutions is to set up a deadline and only the clients that can upload their local models before the deadline would be selected in the FL process. This solution may lead to a slow convergence rate and global model overfitting issues due to the limited client selection. In this paper, we propose the Latency awarE Semi-synchronous client Selection and mOdel aggregation for federated learNing (LESSON) method that allows all the clients to participate in the whole FL process but with different frequencies. That is, faster clients would be scheduled to upload their models more frequently than slow clients, thus resolving the straggler problem and accelerating the convergence speed, while avoiding model overfitting. Also, LESSON is capable of adjusting the tradeoff between the model accuracy and convergence rate by varying the deadline. Extensive simulations have been conducted to compare the performance of LESSON with the other two baseline methods, i.e., FedAvg and FedCS. The simulation results demonstrate that LESSON achieves faster convergence speed than FedAvg and FedCS, and higher model accuracy than FedCS.

FLASH: Federated Learning Across Simultaneous Heterogeneities

FLASH: Heterogeneity-Aware Federated Learning at Scale

HiFlash: Communication-Efficient Hierarchical Federated Learning with Adaptive Staleness Control and Heterogeneity-aware Client-Edge Association

FLASH-RL: Federated Learning Addressing System and Static Heterogeneity using Reinforcement Learning

Heterogeneity-Aware Federated Learning with Adaptive Client Selection and Gradient Compression.

FedSCS: Client Selection for Federated Learning under System Heterogeneity and Client Fairness with a Stackelberg Game Approach.

Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning

Federated Learning Client Selection Mechanism under System and Data Heterogeneity

HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

Communication Efficient Federated Learning With Heterogeneous Structured Client Models

AdaptFL: Adaptive Federated Learning Framework for Heterogeneous Devices

An Aggregation-Free Federated Learning for Tackling Data Heterogeneity

Accelerating Federated Learning via Sequential Training of Grouped Heterogeneous Clients

Latency Aware Semi-synchronous Client Selection and Model Aggregation for Wireless Federated Learning

Flexible Clustered Federated Learning for Client-Level Data Distribution Shift

Clustered Federated Learning in Heterogeneous Environment.

Communication-Efficient Federated Learning With Data and Client Heterogeneity

HSFL: an Efficient Split Federated Learning Framework Via Hierarchical Organization.

AMFL: Asynchronous Multi-level Federated Learning with Client Selection

FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregation

CDFed: Contribution-based Dynamic Federated Learning for Managing System and Statistical Heterogeneity