Abstract:Online federated learning (FL) enables geographically distributed devices to learn a global shared model from locally available streaming data. Most online FL literature considers a best-case scenario regarding the participating clients and the communication channels. However, these assumptions are often not met in real-world applications. Asynchronous settings can reflect a more realistic environment, such as heterogeneous client participation due to available computational power and battery constraints, as well as delays caused by communication channels or straggler devices. Further, in most applications, energy efficiency must be taken into consideration. Using the principles of partial-sharing-based communications, we propose a communication-efficient asynchronous online federated learning (PAO-Fed) strategy. By reducing the communication overhead of the participants, the proposed method renders participation in the learning task more accessible and efficient. In addition, the proposed aggregation mechanism accounts for random participation, handles delayed updates and mitigates their effect on accuracy. We prove the first and second-order convergence of the proposed PAO-Fed method and obtain an expression for its steady-state mean square deviation. Finally, we conduct comprehensive simulations to study the performance of the proposed method on both synthetic and real-life datasets. The simulations reveal that in asynchronous settings, the proposed PAO-Fed is able to achieve the same convergence properties as that of the online federated stochastic gradient while reducing the communication overhead by 98 percent.

A Communication-Efficient Adaptive Algorithm for Federated Learning Under Cumulative Regret

Distributed Online Learning for Joint Regret with Communication Constraints

Communication-Efficient Regret-Optimal Distributed Online Convex Optimization

Adaptive Batchsize Selection and Gradient Compression for Wireless Federated Learning

Communication Compression for Distributed Learning without Control Variates

Federated Online and Bandit Convex Optimization

Lazily Aggregated Quantized Gradient Innovation for Communication-Efficient Federated Learning.

Adaptive Federated Dropout: Improving Communication Efficiency and Generalization for Federated Learning

Asynchronous Online Federated Learning with Reduced Communication Requirements

FedSULP: A Communication-Efficient Federated Learning Framework with Selective Updating and Loss Penalization

Toward Communication Efficient Adaptive Gradient Method

Federated Q-Learning with Reference-Advantage Decomposition: Almost Optimal Regret and Logarithmic Communication Cost

Efficient Methods for Non-stationary Online Learning

Federated Optimization Based on Compression and Event-triggered Communication

Federated Learning: Strategies for Improving Communication Efficiency

Communication-Efficient ADMM-based Federated Learning

Noise-Robust and Resource-Efficient ADMM-based Federated Learning

Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization

Communication-Efficient Federated Group Distributionally Robust Optimization

Communication-Efficient Federated Learning with Accelerated Client Gradient

Communication-Efficient Federated Learning: A Variance-Reduced Stochastic Approach with Adaptive Sparsification.