Abstract:Federated Learning (FL) endeavors to harness decentralized data while preserving privacy, facing challenges of performance, scalability, and collaboration. Asynchronous Federated Learning (AFL) methods have emerged as promising alternatives to their synchronous counterparts bounded by the slowest agent, yet they add additional challenges in convergence guarantees, fairness with respect to compute heterogeneity, and incorporation of staleness in aggregated updates. Specifically, AFL biases model training heavily towards agents who can produce updates faster, leaving slower agents behind, who often also have differently distributed data which is not learned by the global model. Naively upweighting introduces incentive issues, where true fast updating agents may falsely report updates at a slower speed to increase their contribution to model training. We introduce FedStaleWeight, an algorithm addressing fairness in aggregating asynchronous client updates by employing average staleness to compute fair re-weightings. FedStaleWeight reframes asynchronous federated learning aggregation as a mechanism design problem, devising a weighting strategy that incentivizes truthful compute speed reporting without favoring faster update-producing agents by upweighting agent updates based on staleness. Leveraging only observed agent update staleness, FedStaleWeight results in more equitable aggregation on a per-agent basis. We both provide theoretical convergence guarantees in the smooth, non-convex setting and empirically compare FedStaleWeight against the commonly used asynchronous FedBuff with gradient averaging, demonstrating how it achieves stronger fairness, expediting convergence to a higher global model accuracy. Finally, we provide an open-source test bench to facilitate exploration of buffered AFL aggregation strategies, fostering further research in asynchronous federated learning paradigms.

Integrating Staleness and Shapley Value Consistency for Efficient K-Asynchronous Federated Learning

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Towards Efficient and Stable K-Asynchronous Federated Learning with Unbounded Stale Gradients on Non-IID Data

Staleness-Controlled Asynchronous Federated Learning: Accuracy and Efficiency Tradeoff

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

FedASMU: Efficient Asynchronous Federated Learning with Dynamic Staleness-aware Model Update

Asynchronous Federated Learning over Wireless Communication Networks.

ShapleyFL: Robust Federated Learning Based on Shapley Value

Enhancing Convergence in Federated Learning: A Contribution-Aware Asynchronous Approach

FedStaleWeight: Buffered Asynchronous Federated Learning with Fair Aggregation via Staleness Reweighting

Enhancing Edge-Assisted Federated Learning with Asynchronous Aggregation and Cluster Pairing

Mitigating System Bias in Resource Constrained Asynchronous Federated Learning Systems

Tackling Intertwined Data and Device Heterogeneities in Federated Learning with Unlimited Staleness

K Asynchronous Federated Learning with Cosine Similarity Based Aggregation on Non-IID Data.

Robust Model Aggregation for Heterogeneous Federated Learning: Analysis and Optimizations

Asynchronous Federated Learning: A Scalable Approach for Decentralized Machine Learning

Robust Asynchronous Federated Learning with Time-weighted and Stale Model Aggregation

STAFL: Staleness-Tolerant Asynchronous Federated Learning on Non-iid Dataset

Achieving Linear Speedup in Asynchronous Federated Learning with Heterogeneous Clients

MSSA-FL: High-Performance Multi-stage Semi-asynchronous Federated Learning with Non-IID Data.

Towards Stagewise and Energy-Accuracy-balanced Client Selection and Resource Allocation over Dynamic Federated Learning Networks