Abstract:The communication bottleneck of over-the-air federated learning (OA-FL) lies in uploading the gradients of local learning models. In this paper, we study the reduction of the communication overhead in the gradients uploading by using the multiple-input multiple-output (MIMO) technique. We propose a novel sparse-coded multiplexing (SCoM) approach that employs sparse-coding compression and MIMO multiplexing to balance the communication overhead and the learning performance of the FL model. We derive an upper bound on the learning performance loss of the SCoM-based MIMO OA-FL scheme by quantitatively characterizing the gradient aggregation error. Based on the analysis results, we show that the optimal number of multiplexed data streams to minimize the upper bound on the FL learning performance loss is given by the minimum of the numbers of transmit and receive antennas. We then formulate an optimization problem for the design of precoding and post-processing matrices to minimize the gradient aggregation error. To solve this problem, we develop a low-complexity algorithm based on alternating optimization (AO) and alternating direction method of multipliers (ADMM), which effectively mitigates the impact of the gradient aggregation error. Numerical results demonstrate the superb performance of the proposed SCoM approach.

What problem does this paper attempt to address?

### Problems Addressed by the Paper The paper aims to address the communication bottleneck issue when deploying Federated Learning (FL) in wireless networks, particularly the communication overhead of uploading local learning model gradients. Specifically: 1. **Communication Bottleneck**: - In traditional federated learning frameworks, edge devices need to upload local gradients to a central server for aggregation. However, as the size of machine learning models increases (e.g., ResNet152 has 60 million parameters, while GPT-3 has 175 billion parameters), the communication resources required to upload these large model gradients become extremely substantial. - Limited bandwidth and time resources make it difficult to support communication among a large number of edge devices. 2. **Proposed Solution**: - The authors propose a novel method called Sparse-Coded Multiplexing (SCoM), which combines sparse coding compression and multiple-input multiple-output (MIMO) multiplexing to balance communication overhead and federated learning model performance. - By using sparse coding to compress gradients, the communication volume is reduced, and MIMO technology is utilized to transmit multiple data streams in parallel through antenna arrays, further lowering communication overhead. - An optimization problem is formulated to design the precoding and post-processing matrices to minimize gradient aggregation error, and a low-complexity algorithm based on Alternating Optimization (AO) and Alternating Direction Method of Multipliers (ADMM) is developed to solve this problem. 3. **Main Contributions**: - Theoretical analysis of the performance loss of the SCoM method in MIMO federated learning is provided, and an upper bound on the learning performance loss is derived. - An algorithm for jointly optimizing the precoding and post-processing matrices is designed, effectively mitigating the impact of gradient aggregation error. - The optimal number of data streams is determined to achieve the best balance between communication overhead and gradient aggregation performance. In summary, the paper aims to reduce the communication overhead of gradient uploads in federated learning by proposing a new sparse-coded multiplexing method, thereby improving the overall system performance.

Over-the-Air Federated Learning Over MIMO Channels: A Sparse-Coded Multiplexing Approach

A Sparse-Coded Multiplexing Approach for MIMO Over-the-Air Federated Learning

Over-the-Air Federated Learning in MIMO Cloud-RAN Systems

Over-the-Air Federated Multi-Task Learning Over MIMO Multiple Access Channels

Decentralized Federated Learning via MIMO Over-the-Air Computation: Consensus Analysis and Performance Optimization

Over-the-Air Decentralized Federated Learning under MIMO Noisy Channel

Multi-Task Federated Learning with Over-the-Air Computation for MIMO Interference Channels

Deep Learning Based Coded Over-the-Air Computation for Personalized Federated Learning

Cloud-RAN Over-the-Air Federated Learning

Digital Over-the-Air Federated Learning in Multi-Antenna Systems

Secure Federated Learning over Wireless Communication Networks with Model Compression

STAR-RIS Integrated Nonorthogonal Multiple Access and Over-the-Air Federated Learning: Framework, Analysis, and Optimization

Simultaneous Federated Learning and Information Transmission over Time-Varying MIMO Channels.

Federated Learning in Multi-RIS-Aided Systems

Joint Antenna Selection and Beamforming for Massive MIMO-enabled Over-the-Air Federated Learning

Secure Federated Learning with Model Compression.

A Random Access Scheme for Federated Learning over Massive MIMO Systems

Uplink Over-the-Air Aggregation for Multi-Model Wireless Federated Learning

Channel-Estimation-Free Gradient Aggregation for Over-the-Air SIMO Federated Learning

Channel and Gradient-Importance Aware Device Scheduling for Over-the-Air Federated Learning