Abstract:Leveraging over-the-air computations for model aggregation is an effective approach to cope with the communication bottleneck in federated edge learning. By exploiting the superposition properties of multi-access channels, this approach facilitates an integrated design of communication and computation, thereby enhancing system privacy while reducing implementation costs. However, the inherent electromagnetic interference in radio channels often exhibits heavy-tailed distributions, giving rise to exceptionally strong noise in globally aggregated gradients that can significantly deteriorate the training performance. To address this issue, we propose a novel gradient clipping method, termed Median Anchored Clipping (MAC), to combat the detrimental effects of heavy-tailed noise. We also derive analytical expressions for the convergence rate of model training with analog over-the-air federated learning under MAC, which quantitatively demonstrates the effect of MAC on training performance. Extensive experimental results show that the proposed MAC algorithm effectively mitigates the impact of heavy-tailed noise, hence substantially enhancing system robustness.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to deal with the heavy - tailed noise problem caused by electromagnetic interference when performing model aggregation via over - the - air computation in Federated Edge Learning. Specifically: 1. **Communication Bottleneck and Privacy Protection**: Utilizing the superposition property of over - the - air computation can effectively address the communication bottleneck problem in Federated Edge Learning, while enhancing system privacy and reducing implementation costs. 2. **Impact of Heavy - Tailed Noise**: However, the inherent electromagnetic interference in wireless channels usually exhibits a heavy - tailed distribution, which will cause extremely strong noise in the global aggregated gradient, thereby significantly degrading the training performance. To solve this problem, the authors propose a new gradient clipping method - Median Anchored Clipping (MAC). The MAC method aims to mitigate the impact of heavy - tailed noise on the training process through the following steps: - **Centralization**: Subtract the median value from each element of the global aggregated gradient to minimize the L - 1 deviation. - **Clipping**: Clip the centralized gradient values and limit their range within a specified threshold. - **Recovery**: Add the median back to each element to restore the gradient information. Through these steps, the MAC method can largely preserve the original gradient information while effectively alleviating the adverse effects brought by heavy - tailed noise. In addition, the authors also derive the convergence rate of simulated over - the - air Federated Learning under the MAC algorithm and verify the effectiveness of the MAC algorithm through a large number of experiments. The experimental results show that the MAC algorithm can significantly improve the robustness and training stability of the system, especially performing excellently under extreme noise conditions. In summary, the main contributions of this paper include: - Proposing a new gradient clipping method MAC for mitigating the impact of heavy - tailed noise. - Deriving the convergence rate formula under the MAC algorithm. - Proving the effectiveness and robustness of the MAC algorithm through experiments. The formulas are as follows: - Definition of median: \[ \text{med}(w)=\text{median}\{w_i, i\in [d]\} \] - Centralization operation: \[ g_k\leftarrow g_k - \text{med}(g_k)\cdot\mathbf{1} \] - Clipping operation: \[ g_{k,i}\leftarrow \text{sgn}(g_{k,i})\cdot\min(|g_{k,i}|, C) \] - Recovery operation: \[ \check{g}_k\leftarrow g_k+\text{med}(g_k)\cdot\mathbf{1} \] These steps together ensure the effectiveness and robustness of the MAC algorithm.

Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping

Deep Learning Based Coded Over-the-Air Computation for Personalized Federated Learning

Analog Gradient Aggregation for Federated Learning Over Wireless Networks: Customized Design and Convergence Analysis

Message Passing Based Wireless Federated Learning Via Analog Message Aggregation

Over-the-air Learning Rate Optimization for Federated Learning

Federated Learning via Over-the-Air Computation

Federated Learning over Wireless Fading Channels

Over-the-Air Federated Learning and Optimization

Federated Learning from Heterogeneous Data via Controlled Air Aggregation with Bayesian Estimation

IRS Assisted Federated Learning A Broadband Over-the-Air Aggregation Approach

Federated Learning from Heterogeneous Data via Controlled Bayesian Air Aggregation

One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis

Federated Learning in Multi-RIS-Aided Systems

IRS Assisted Federated Learning: A Broadband Over-the-Air Aggregation Approach

Convergence Analysis and Optimization of Over-the-Air Federated Meta-Learning.

Over-the-Air Federated Learning via Weighted Aggregation

Edge Federated Learning Via Unit-Modulus Over-The-Air Computation

Accuracy-Security Tradeoff with Balanced Aggregation and Artificial Noise for Wireless Federated Learning

Over-the-Air Computation Empowered Federated Learning: A Joint Uplink-Downlink Design

Movable Antenna-Aided Federated Learning with Over-the-Air Aggregation: Joint Optimization of Positioning, Beamforming, and User Selection

Gradient and Channel Aware Dynamic Scheduling for Over-the-Air Computation in Federated Edge Learning Systems