Abstract:Decentralized federated learning (DFL), inherited from distributed optimization, is an emerging paradigm to leverage the explosively growing data from wireless devices in a fully distributed manner.DFL enables joint training of machine learning model under device to device (D2D) communication fashion without the coordination of a parameter server. However, the deployment of wireless DFL is facing some pivotal challenges. Communication is a critical bottleneck due to the required extensive message exchange between neighbor devices to share the learned model. Besides, consensus becomes increasingly difficult as the number of devices grows because there is no available central server to perform coordination. To overcome these difficulties, this paper proposes employing over-the-air computation (Aircomp) to improve communication efficiency by exploiting the superposition property of analog waveform in multi-access channels, and introduce the mixing matrix mechanism to promote consensus using the spectral property of symmetric doubly stochastic matrix. Specifically, we develop a novel multiple-input multiple-output over-the-air DFL (MIMO OA-DFL) framework to study over-the-air DFL problem over MIMO multiple access channels. We conduct a general convergence analysis to quantitatively capture the influence of aggregation weight and communication error on the MIMO OA-DFL performance in \emph{ad hoc} networks. The result shows that the communication error together with the spectral gap of mixing matrix has a significant impact on the learning performance. Based on this, a joint communication-learning optimization problem is formulated to optimize transceiver beamformers and mixing matrix. Extensive numerical experiments are performed to reveal the characteristics of different topologies and demonstrate the substantial learning performance enhancement of our proposed algorithm.

One-Bit Aggregation for Over-the-Air Federated Learning Against Byzantine Attacks

Over-The-Air Federated Learning under Byzantine Attacks

Byzantine-Resilient Federated Machine Learning via Over-the-Air Computation

Message Passing Based Wireless Federated Learning Via Analog Message Aggregation

One-Bit Byzantine-Tolerant Distributed Learning via Over-the-Air Computation

Accuracy-Security Tradeoff with Balanced Aggregation and Artificial Noise for Wireless Federated Learning

BEV-SGD: Best Effort Voting SGD for Analog Aggregation Based Federated Learning against Byzantine Attackers

Secure Federated Learning over Wireless Communication Networks with Model Compression

Secure Federated Learning with Model Compression.

Split Aggregation: Lightweight Privacy-Preserving Federated Learning Resistant to Byzantine Attacks

One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis

Approximate Wireless Communication for Lossy Gradient Updates in IoT Federated Learning

Over-the-Air Federated Learning In Broadband Communication

Over-the-Air Decentralized Federated Learning under MIMO Noisy Channel

Federated Learning Robust to Byzantine Attacks: Achieving Zero Optimality Gap

IRS Assisted Federated Learning A Broadband Over-the-Air Aggregation Approach

Decentralized Federated Learning via MIMO Over-the-Air Computation: Consensus Analysis and Performance Optimization

Quantization Bits Allocation for Wireless Federated Learning

Approximate Wireless Communication for Federated Learning

Over-The-Air Federated Learning: Status Quo, Open Challenges, and Future Directions

Toward Secure and Private Over-the-Air Federated Learning