Abstract:Federated Learning (FL) is an emerging distributed learning paradigm with the privacy-preserving advantage of collaboratively training a shared model across multiple participants. Considering the prevailing device heterogeneity circumstance in practice, asynchronous interaction is introduced into FL to break the straggler barrier of synchronization, at the cost of significant accuracy degradation derived from model staleness. Although quite a few works attempt to partially mitigate the detrimental impact after occurring staleness issue, they neglect to control the overall staleness degree of clients-side local models from the whole training perspective, resulting in highly-stale models for aggregation and slow convergence speed. To this end, we propose a Staleness-Controlled Asynchronous Federated Learning (SC-AFL) method, which enables to restrict staleness degree of local models within a certain bound via dynamically tuning the aggregated strategy of each round, aiming to strike a good balance between accuracy guarantee and convergence acceleration. Specifically, we leverage the Lyapunov optimization framework to decouple the troublesome round-coupling problem into the single-round sequential solving problem, and further develop a deterministic algorithm that selects the aggregated number of clients to minimize training time under the constraint of maintaining staleness queue stability. Besides, we derive the theoretical convergence analysis of SC-AFL and also present the upper bound of the performance gap with the optimum. Extensive experiments on three datasets demonstrate the superiority of SC-AFL in terms of time-to-accuracy speedup on both IID and Non-IID data distributions, achieving a good balance between model accuracy and convergence efficiency in AFL system.

FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Update

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

Asynchronous Federated Learning over Wireless Communication Networks.

FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systems

Integrating Staleness and Shapley Value Consistency for Efficient K-Asynchronous Federated Learning

AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices

Staleness-Controlled Asynchronous Federated Learning: Accuracy and Efficiency Tradeoff

Federated learning with workload-aware client scheduling in heterogeneous systems

FedSA: A Semi-Asynchronous Federated Learning Mechanism in Heterogeneous Edge Computing

Asynchronous Online Federated Learning for Edge Devices with Non-IID Data

Mitigating System Bias in Resource Constrained Asynchronous Federated Learning Systems

Tackling Intertwined Data and Device Heterogeneities in Federated Learning with Unlimited Staleness

Towards addressing aggregation deviation for model training in resource-scarce edge environment

STAFL: Staleness-Tolerant Asynchronous Federated Learning on Non-iid Dataset

ASMAFL: Adaptive Staleness-aware Momentum Asynchronous Federated Learning in Edge Computing

FedAST: Federated Asynchronous Simultaneous Training

FedLC: Accelerating Asynchronous Federated Learning in Edge Computing

Enhancing Convergence in Federated Learning: A Contribution-Aware Asynchronous Approach

Takle the Data Heterogeneous Variations in Federated Learning Via Adaptive Aggregate Weights

Enhancing Edge-Assisted Federated Learning with Asynchronous Aggregation and Cluster Pairing

Robust Asynchronous Federated Learning with Time-weighted and Stale Model Aggregation