Abstract:Federated Learning (FL) endeavors to harness decentralized data while preserving privacy, facing challenges of performance, scalability, and collaboration. Asynchronous Federated Learning (AFL) methods have emerged as promising alternatives to their synchronous counterparts bounded by the slowest agent, yet they add additional challenges in convergence guarantees, fairness with respect to compute heterogeneity, and incorporation of staleness in aggregated updates. Specifically, AFL biases model training heavily towards agents who can produce updates faster, leaving slower agents behind, who often also have differently distributed data which is not learned by the global model. Naively upweighting introduces incentive issues, where true fast updating agents may falsely report updates at a slower speed to increase their contribution to model training. We introduce FedStaleWeight, an algorithm addressing fairness in aggregating asynchronous client updates by employing average staleness to compute fair re-weightings. FedStaleWeight reframes asynchronous federated learning aggregation as a mechanism design problem, devising a weighting strategy that incentivizes truthful compute speed reporting without favoring faster update-producing agents by upweighting agent updates based on staleness. Leveraging only observed agent update staleness, FedStaleWeight results in more equitable aggregation on a per-agent basis. We both provide theoretical convergence guarantees in the smooth, non-convex setting and empirically compare FedStaleWeight against the commonly used asynchronous FedBuff with gradient averaging, demonstrating how it achieves stronger fairness, expediting convergence to a higher global model accuracy. Finally, we provide an open-source test bench to facilitate exploration of buffered AFL aggregation strategies, fostering further research in asynchronous federated learning paradigms.

STAFL: Staleness-Tolerant Asynchronous Federated Learning on Non-iid Dataset

Asynchronous Federated Learning over Wireless Communication Networks.

FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Update

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

Integrating Staleness and Shapley Value Consistency for Efficient K-Asynchronous Federated Learning

Towards Efficient Asynchronous Federated Learning in Heterogeneous Edge Environments

Asynchronous Online Federated Learning for Edge Devices with Non-IID Data

Stochastic Clustered Federated Learning

Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Data

FedStaleWeight: Buffered Asynchronous Federated Learning with Fair Aggregation via Staleness Reweighting

Enhancing Edge-Assisted Federated Learning with Asynchronous Aggregation and Cluster Pairing

Latency Aware Semi-synchronous Client Selection and Model Aggregation for Wireless Federated Learning

CSAFL: A Clustered Semi-Asynchronous Federated Learning Framework

An Efficient Asynchronous Federated Learning Protocol for Edge Devices

Blockchain-Enabled Asynchronous Federated Learning in Edge Computing

AFL-DCS: An asynchronous federated learning framework with dynamic client scheduling

Stragglers-Aware Low-Latency Synchronous Federated Learning via Layer-Wise Model Updates

Federated learning with workload-aware client scheduling in heterogeneous systems

Time-triggered Federated Learning over Wireless Networks

Enhancing Convergence in Federated Learning: A Contribution-Aware Asynchronous Approach

Towards addressing aggregation deviation for model training in resource-scarce edge environment