Abstract:Federated learning (FL) can suffer from a communication bottleneck when deployed in mobile networks, limiting participating clients and deterring FL convergence. The impact of practical air interfaces with discrete modulations on FL has not previously been studied in depth. This paper proposes a new paradigm of flexible aggregation-based FL (F$^2$L) over orthogonal frequency division multiple-access (OFDMA) air interface, termed as ``OFDMA-F$^2$L'', allowing selected clients to train local models for various numbers of iterations before uploading the models in each aggregation round. We optimize the selections of clients, subchannels and modulations, adapting to channel conditions and computing powers. Specifically, we derive an upper bound on the optimality gap of OFDMA-F$^2$L capturing the impact of the selections, and show that the upper bound is minimized by maximizing the weighted sum rate of the clients per aggregation round. A Lagrange-dual based method is developed to solve this challenging mixed integer program of weighted sum rate maximization, revealing that a ``winner-takes-all'' policy provides the almost surely optimal client, subchannel, and modulation selections. Experiments on multilayer perceptrons and convolutional neural networks show that OFDMA-F$^2$L with optimal selections can significantly improve the training convergence and accuracy, e.g., by about 18\% and 5\%, compared to potential alternatives.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the communication bottleneck problem faced when deploying Federated Learning (FL) in mobile networks. Specifically, when FL is applied in wireless networks, due to the limited number of participating clients and the slow convergence speed of FL, a communication bottleneck is caused. In addition, the impact of the actual air interface (such as Orthogonal Frequency - Division Multiple Access, OFDMA) using discrete modulation on FL has not been deeply studied. This paper proposes a new FL paradigm based on flexible aggregation - "OFDMA - F2L", which allows selected clients to perform different numbers of local model training iterations before each aggregation round and then upload the model. By optimizing the selection of clients, sub - channels and modulation, and adapting to channel conditions and computing capabilities, the communication efficiency and training performance of FL are improved. The main contributions of the paper include: 1. Proposing the OFDMA - F2L method, in which selected clients can perform local model training according to their available computing capabilities and time, and can simultaneously upload local models on different sub - channels, increasing the number of participating clients and prolonging their training time. 2. Analyzing the upper bound of convergence of OFDMA - F2L and quantifying the impact of client, sub - channel and modulation selection on convergence. 3. Minimizing the upper bound of convergence by jointly optimizing the selection of clients, sub - channels and modulation, which is transformed into maximizing the weighted total rate of clients. 4. Designing a method based on Lagrange duality and finding that the "winner - takes - all" strategy can almost determine the optimal selection of clients, sub - channels and modulation to achieve the rapid convergence of OFDMA - F2L. Experimental results show that, compared with partial optimal selection or synchronous FL, the convergence speed and training accuracy of OFDMA - F2L on Multi - Layer Perceptron (MLP) and Convolutional Neural Network (CNN) models are significantly improved. For example, they are respectively increased by about 18% and 5% on the MLP model.

OFDMA-F$^2$L: Federated Learning With Flexible Aggregation Over an OFDMA Air Interface

OFDMA-F²L: Federated Learning with Flexible Aggregation over an OFDMA Air Interface

OFDMA-F^2L: Federated Learning with Flexible Aggregation over an OFDMA Air Interface

Message Passing Based Wireless Federated Learning Via Analog Message Aggregation

Fast Convergence for Federated Learning in OFDMA Systems

Over-the-air Learning Rate Optimization for Federated Learning

Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costs

Uplink Over-the-Air Aggregation for Multi-Model Wireless Federated Learning

Asynchronous Federated Learning over Wireless Communication Networks.

Decentralized Federated Learning via MIMO Over-the-Air Computation: Consensus Analysis and Performance Optimization

Movable Antenna-Aided Federated Learning with Over-the-Air Aggregation: Joint Optimization of Positioning, Beamforming, and User Selection

Over-the-Air Federated Multi-Task Learning Over MIMO Multiple Access Channels

Beamforming and Device Selection Design in Federated Learning with Over-the-air Aggregation

Federated Learning from Heterogeneous Data via Controlled Bayesian Air Aggregation

Federated Learning from Heterogeneous Data via Controlled Air Aggregation with Bayesian Estimation

Federated Learning via Over-the-Air Computation

Over-the-Air Federated Learning and Optimization

Quantization Bits Allocation for Wireless Federated Learning

Non-Orthogonal Multiple Access Assisted Federated Learning via Wireless Power Transfer: A Cost-Efficient Approach

Communication-Efficient Model Aggregation with Layer Divergence Feedback in Federated Learning

Federated Learning with Flexible Architectures