Abstract:Federated learning (FL) involves training machine learning models over distributed edge nodes (i.e., workers) while facing three critical challenges, edge heterogeneity, Non-IID data and communication resource constraint. In the synchronous FL, the parameter server has to wait for the slowest workers, leading to significant waiting time due to edge heterogeneity. Though asynchronous FL can well tackle the edge heterogeneity, it requires frequent model transfers, resulting in massive communication resource consumption. Moreover, the different relative frequency of workers participating in asynchronous updating may seriously hurt training accuracy, especially on Non-IID data. In this paper, we propose a semi-asynchronous federated learning mechanism (FedSA), where the parameter server aggregates a certain number of local models by their arrival order in each round. We theoretically analyze the quantitative relationship between the convergence bound of FedSA and different factors, e.g., the number of participating workers in each round, the degree of data Non-IID and edge heterogeneity. Based on the convergence bound, we present an efficient algorithm to determine the number of participating workers to minimize the training completion time. To further improve the training accuracy on Non-IID data, FedSA deploys adaptive learning rates for workers by their relative participation frequency. We extend our proposed mechanism to the dynamic and multiple learning tasks scenarios. Experimental results on the testbed show that our proposed mechanism and algorithms address the three challenges more effectively than the state-of-the-art solutions.

ESync: Accelerating Intra-Domain Federated Learning in Heterogeneous Data Centers

AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices

Towards Efficient Asynchronous Federated Learning in Heterogeneous Edge Environments

FedLC: Accelerating Asynchronous Federated Learning in Edge Computing

Asynchronous Federated Learning over Wireless Communication Networks.

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

FedGSync: Jointly Optimized Weak Synchronization and Gradient Transmission for Fast Distributed Machine Learning in Heterogeneous WAN

Arena: A Learning-based Synchronization Scheme for Hierarchical Federated Learning--Technical Report

Partial Synchronization to Accelerate Federated Learning over Relay-Assisted Edge Networks

Asynchronous Decentralized Federated Learning for Heterogeneous Devices

FedSA: A Semi-Asynchronous Federated Learning Mechanism in Heterogeneous Edge Computing

Enhancing Edge-Assisted Federated Learning with Asynchronous Aggregation and Cluster Pairing

Federated learning with workload-aware client scheduling in heterogeneous systems

Enhancing Federated Learning Convergence with Dynamic Data Queue and Data Entropy-driven Participant Selection

Asynchronous Federated Learning on Heterogeneous Devices: A Survey

Enhancing Convergence in Federated Learning: A Contribution-Aware Asynchronous Approach

Adaptive Configuration for Heterogeneous Participants in Decentralized Federated Learning

Lazy Aggregation for Heterogeneous Federated Learning

No One Idles: Efficient Heterogeneous Federated Learning with Parallel Edge and Server Computation.

Heterogeneity-Aware Federated Learning with Adaptive Local Epoch Size in Edge Computing

Accelerating Federated Learning with Data and Model Parallelism in Edge Computing