Abstract:Decentralized federated learning (DFL), inherited from distributed optimization, is an emerging paradigm to leverage the explosively growing data from wireless devices in a fully distributed manner.DFL enables joint training of machine learning model under device to device (D2D) communication fashion without the coordination of a parameter server. However, the deployment of wireless DFL is facing some pivotal challenges. Communication is a critical bottleneck due to the required extensive message exchange between neighbor devices to share the learned model. Besides, consensus becomes increasingly difficult as the number of devices grows because there is no available central server to perform coordination. To overcome these difficulties, this paper proposes employing over-the-air computation (Aircomp) to improve communication efficiency by exploiting the superposition property of analog waveform in multi-access channels, and introduce the mixing matrix mechanism to promote consensus using the spectral property of symmetric doubly stochastic matrix. Specifically, we develop a novel multiple-input multiple-output over-the-air DFL (MIMO OA-DFL) framework to study over-the-air DFL problem over MIMO multiple access channels. We conduct a general convergence analysis to quantitatively capture the influence of aggregation weight and communication error on the MIMO OA-DFL performance in \emph{ad hoc} networks. The result shows that the communication error together with the spectral gap of mixing matrix has a significant impact on the learning performance. Based on this, a joint communication-learning optimization problem is formulated to optimize transceiver beamformers and mixing matrix. Extensive numerical experiments are performed to reveal the characteristics of different topologies and demonstrate the substantial learning performance enhancement of our proposed algorithm.

On-Device Federated Learning Via Second-Order Optimization With Over-The-Air Computation

Optimize Scheduling of Federated Learning on Battery-powered Mobile Devices

Federated Learning via Over-the-Air Computation

Decentralized Over-the-Air Federated Learning by Second-Order Optimization Method

Over-the-air Learning Rate Optimization for Federated Learning

Scalable and Resource-Efficient Second-Order Federated Learning via Over-the-Air Aggregation

Over-the-Air Federated Learning and Optimization

Asynchronous Federated Learning over Wireless Communication Networks.

Hierarchical federated learning based on wireless D2D networks

Federated Learning in Multi-RIS-Aided Systems

Decentralized Federated Learning via MIMO Over-the-Air Computation: Consensus Analysis and Performance Optimization

Learning Rate Optimization for Federated Learning Exploiting Over-the-air Computation

Convergence Analysis and Optimization of Over-the-Air Federated Meta-Learning.

Online Optimization for Over-the-Air Federated Learning with Energy Harvesting

Edge-Based Communication Optimization for Distributed Federated Learning

Boosting Fairness and Robustness in Over-the-Air Federated Learning

Over-the-Air Hierarchical Personalized Federated Learning

Over-the-Air Computation Empowered Federated Learning: A Joint Uplink-Downlink Design

Federated Learning over Wireless Networks: Optimization Model Design and Analysis

Over-the-Air Computation Aided Federated Learning with the Aggregation of Normalized Gradient.

Multiple Parallel Federated Learning Via Over-the-Air Computation