Abstract:We study federated machine learning at the wireless network edge, where limited power wireless devices, each with its own dataset, build a joint model with the help of a remote parameter server (PS). We consider a bandwidth-limited fading multiple access channel (MAC) from the wireless devices to the PS, and propose various techniques to implement distributed stochastic gradient descent (DSGD). We first propose a digital DSGD (D-DSGD) scheme, in which one device is selected opportunistically for transmission at each iteration based on the channel conditions; the scheduled device quantizes its gradient estimate to a finite number of bits imposed by the channel condition, and transmits these bits to the PS in a reliable manner. Next, motivated by the additive nature of the wireless MAC, we propose a novel analog communication scheme, referred to as the compressed analog DSGD (CA-DSGD), where the devices first sparsify their gradient estimates while accumulating error, and project the resultant sparse vector into a low-dimensional vector for bandwidth reduction. Numerical results show that D-DSGD outperforms other digital approaches in the literature; however, in general the proposed CA-DSGD algorithm converges faster than the D-DSGD scheme and other schemes in the literature, and reaches a higher level of accuracy. We have observed that the gap between the analog and digital schemes increases when the datasets of devices are not independent and identically distributed (i.i.d.). Furthermore, the performance of the CA-DSGD scheme is shown to be robust against imperfect channel state information (CSI) at the devices. Overall these results show clear advantages for the proposed analog over-the-air DSGD scheme, which suggests that learning and communication algorithms should be designed jointly to achieve the best end-to-end performance in machine learning applications at the wireless edge.

Fair Wireless Federated Learning Through the Identification of a Common Descent Direction

AdaFed: Fair Federated Learning via Adaptive Common Descent Direction

Asynchronous Federated Learning over Wireless Communication Networks.

Addressing Algorithmic Disparity and Performance Inconsistency in Federated Learning

Multi-Dimensional Fair Federated Learning

Federated Learning over Wireless Fading Channels

Federated Learning for Non-IID Data Via Unified Feature Learning and Optimization Objective Alignment

Fairness and Accuracy in Federated Learning

A Joint Learning and Communications Framework for Federated Learning over Wireless Networks

Online-Score-Aided Federated Learning: Taming the Resource Constraints in Wireless Networks

Personalized Federated Learning towards Communication Efficiency, Robustness and Fairness

Federated Learning over Wireless Device-to-Device Networks: Algorithms and Convergence Analysis

Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing

Fairness-Aware Client Selection for Federated Learning

Data Collaborative Federated Learning for Non-i.i.d Data in Wireless Networks

FedHF: A High Fairness Federated Learning Algorithm Based on Deconfliction in Heterogeneous Networks.

Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity

Online Client Selection for Asynchronous Federated Learning With Fairness Consideration

Fair Concurrent Training of Multiple Models in Federated Learning

Over-The-Air Clustered Wireless Federated Learning

WassFFed: Wasserstein Fair Federated Learning