Abstract:We study federated machine learning at the wireless network edge, where limited power wireless devices, each with its own dataset, build a joint model with the help of a remote parameter server (PS). We consider a bandwidth-limited fading multiple access channel (MAC) from the wireless devices to the PS, and propose various techniques to implement distributed stochastic gradient descent (DSGD). We first propose a digital DSGD (D-DSGD) scheme, in which one device is selected opportunistically for transmission at each iteration based on the channel conditions; the scheduled device quantizes its gradient estimate to a finite number of bits imposed by the channel condition, and transmits these bits to the PS in a reliable manner. Next, motivated by the additive nature of the wireless MAC, we propose a novel analog communication scheme, referred to as the compressed analog DSGD (CA-DSGD), where the devices first sparsify their gradient estimates while accumulating error, and project the resultant sparse vector into a low-dimensional vector for bandwidth reduction. Numerical results show that D-DSGD outperforms other digital approaches in the literature; however, in general the proposed CA-DSGD algorithm converges faster than the D-DSGD scheme and other schemes in the literature, and reaches a higher level of accuracy. We have observed that the gap between the analog and digital schemes increases when the datasets of devices are not independent and identically distributed (i.i.d.). Furthermore, the performance of the CA-DSGD scheme is shown to be robust against imperfect channel state information (CSI) at the devices. Overall these results show clear advantages for the proposed analog over-the-air DSGD scheme, which suggests that learning and communication algorithms should be designed jointly to achieve the best end-to-end performance in machine learning applications at the wireless edge.

Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping

Deep Learning Based Coded Over-the-Air Computation for Personalized Federated Learning

Analog Gradient Aggregation for Federated Learning Over Wireless Networks: Customized Design and Convergence Analysis

Message Passing Based Wireless Federated Learning Via Analog Message Aggregation

Over-the-air Learning Rate Optimization for Federated Learning

Federated Learning via Over-the-Air Computation

Federated Learning over Wireless Fading Channels

Over-the-Air Federated Learning and Optimization

Federated Learning from Heterogeneous Data via Controlled Air Aggregation with Bayesian Estimation

IRS Assisted Federated Learning A Broadband Over-the-Air Aggregation Approach

Federated Learning from Heterogeneous Data via Controlled Bayesian Air Aggregation

Robust Federated Learning with Noisy Communication.

One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis

IRS Assisted Federated Learning: A Broadband Over-the-Air Aggregation Approach

Convergence Analysis and Optimization of Over-the-Air Federated Meta-Learning.

Coded Over-the-Air Computation for Model Aggregation in Federated Learning

Over-the-Air Federated Learning via Weighted Aggregation

Edge Federated Learning Via Unit-Modulus Over-The-Air Computation

Accuracy-Security Tradeoff with Balanced Aggregation and Artificial Noise for Wireless Federated Learning

Over-the-Air Computation Empowered Federated Learning: A Joint Uplink-Downlink Design

Jamming-Resilient Over-the-Air Computation for Federated Learning in Edge Intelligence.