Abstract:Federated learning (FL) is a machine learning paradigm where a shared central model is learned across distributed edge devices while the training data remains on these devices. Federated Averaging (FedAvg) is the leading optimization method for training non-convex models in this setting with a synchronized protocol. However, the assumptions made by FedAvg are not realistic given the heterogeneity of devices. In particular, the volume and distribution of collected data vary in the training process due to different sampling rates of edge devices. The edge devices themselves also vary in their available communication bandwidth and system configurations, such as memory, processor speed, and power requirements. This leads to vastly different training times as well as model/data transfer times. Furthermore, availability issues at edge devices can lead to a lack of contribution from specific edge devices to the federated model. In this paper, we present an Asynchronous Online Federated Learning (ASO-Fed) framework, where the edge devices perform online learning with continuous streaming local data and a central server aggregates model parameters from clients. Our framework updates the central model in an asynchronous manner to tackle the challenges associated with both varying computational loads at heterogeneous edge devices and edge devices that lag behind or dropout. We perform extensive experiments on a simulated benchmark image dataset and three real-world non-IID streaming datasets. The results demonstrate the effectiveness of \model~on converging fast and maintaining good prediction performance.

Gradient Scheduling with Global Momentum for Asynchronous Federated Learning in Edge Environment

Energy Optimization for Federated Learning on Consumer Mobile Devices with Asynchronous SGD and Application Co-Execution

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

Gradient-Congruity Guided Federated Sparse Training

Data and Channel-Adaptive Sensor Scheduling for Federated Edge Learning via Over-the-Air Gradient Aggregation

Accelerating Federated Learning with Adaptive Extra Local Updates Upon Edge Networks

Towards Efficient Asynchronous Federated Learning in Heterogeneous Edge Environments

FedGSync: Jointly Optimized Weak Synchronization and Gradient Transmission for Fast Distributed Machine Learning in Heterogeneous WAN

FEDERATED STOCHASTIC GRADIENT DESCENT BEGETS SELF-INDUCED MOMENTUM

Asynchronous Decentralized Federated Learning for Heterogeneous Devices

Momentum Benefits Non-iid Federated Learning Simply and Provably

Effective Federated Adaptive Gradient Methods with Non-IID Decentralized Data

A Derivative-Incorporated Adaptive Gradient Method for Federated Learning

Asynchronous Local-SGD Training for Language Modeling

Faster On-Device Training Using New Federated Momentum Algorithm

Scheduling for Cellular Federated Edge Learning With Importance and Channel Awareness

Heterogeneity-Aware Federated Learning with Adaptive Local Epoch Size in Edge Computing

Ordered Momentum for Asynchronous SGD

FedLC: Accelerating Asynchronous Federated Learning in Edge Computing

Joint Dynamic Grouping and Gradient Coding for Time-Critical Distributed Machine Learning in Heterogeneous Edge Networks

Asynchronous Online Federated Learning for Edge Devices with Non-IID Data