Abstract:Spiking Neural Networks (SNNs) have recently gained significant interest in on-chip learning in embedded devices and emerged as an energy-efficient alternative to conventional Artificial Neural Networks (ANNs). However, to extend SNNs to a Federated Learning (FL) setting involving collaborative model training, the communication between the local devices and the remote server remains the bottleneck, which is often restricted and costly. In this paper, we first explore the inherent robustness of SNNs under noisy communication in FL. Building upon this foundation, we propose a novel Federated Learning with Top-K Sparsification (FLTS) algorithm to reduce the bandwidth usage for FL training. We discover that the proposed scheme with SNNs allows more bandwidth savings compared to ANNs without impacting the model's accuracy. Additionally, the number of parameters to be communicated can be reduced to as low as 6 percent of the size of the original model. We further improve the communication efficiency by enabling dynamic parameter compression during model training. Extensive experiment results demonstrate that our proposed algorithms significantly outperform the baselines in terms of communication cost and model accuracy and are promising for practical network-efficient FL with SNNs.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are the communication efficiency and noise robustness issues when using Spiking Neural Networks (SNNs) in the Federated Learning (FL) environment. Specifically, the research focuses on the following aspects: 1. **Communication Bottleneck Problem**: In federated learning, the frequent communication between local devices and remote servers is the main bottleneck, especially in large - scale deployments, which will lead to high communication costs. 2. **Impact of Noisy Communication**: Communication networks in the real world are usually noisy and prone to packet loss and transmission errors, which will have a negative impact on the convergence and accuracy of the model. 3. **Robustness of SNN**: Compared with traditional Artificial Neural Networks (ANNs), SNNs perform more robustly in noisy communication environments. Therefore, this characteristic can be utilized to improve the communication efficiency of the federated learning system. To address the above problems, the paper proposes the following solutions: - **Explore the Noise Robustness of SNN**: Through experiments to verify the stability of SNN under different noise levels, it is found that SNN is more robust than ANN. - **Propose the Top - κ Sparsification Algorithm (FLTS)**: Based on the robustness of SNN, a new federated learning algorithm - Federated Learning with Top - κ Sparsification (FLTS) is designed to reduce the use of communication bandwidth. This algorithm only transmits the parameters with the largest gradients, thereby significantly reducing the communication volume. - **Dynamic Compression Rate Adjustment (FLDR)**: Further propose an improved algorithm - Federated Learning with Dynamic - κ Reduction (FLDR). This algorithm dynamically adjusts the compression rate during the training process, thereby further reducing the communication overhead. The paper verifies the effectiveness of these methods through extensive experiments. The results show that, compared with traditional methods, the newly proposed algorithms can significantly reduce communication costs while maintaining model accuracy and show better robustness in noisy communication environments. In summary, this paper aims to solve the communication bottleneck problem and improve the overall performance of the federated learning system by studying the robustness of SNN in noisy communication environments and proposing efficient federated learning algorithms.

The Robustness of Spiking Neural Networks in Communication and its Application towards Network Efficiency in Federated Learning

Communication Trade-offs in Federated Learning of Spiking Neural Networks

Message Passing Based Wireless Federated Learning Via Analog Message Aggregation

Efficient Structure Slimming for Spiking Neural Networks

Efficient Asynchronous Federated Neuromorphic Learning of Spiking Neural Networks.

Efficient Model Compression via Global Sparsification for Over-the-Air Federated Learning.

Asynchronous Federated Learning over Wireless Communication Networks.

Communication-Efficient Federated Learning with Binary Neural Networks

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Accelerating Split Federated Learning over Wireless Communication Networks

Spiking Neural Networks in Vertical Federated Learning: Performance Trade-offs

SNN-SC: A Spiking Semantic Communication Framework for Collaborative Intelligence

Spiking Neural Networks -- Part III: Neuromorphic Communications

Communication-Efficient Federated Learning via Regularized Sparse Random Networks

Heterogeneous Federated Learning with Convolutional and Spiking Neural Networks

Recent Advances in Scalable Energy-Efficient and Trustworthy Spiking Neural networks: from Algorithms to Technology

Comprehensive Online Training and Deployment for Spiking Neural Networks

Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Better

Adaptive Model Pruning and Personalization for Federated Learning Over Wireless Networks