Abstract:To provide a flexible tradeoff between inference accuracy and resource requirement at runtime, the slimmable neural network (SNN), a single network executable at different widths with the same deploying and management cost as that of a single model, has been proposed. However, how to effectively train SNN among massive devices in edge computing without revealing their local data remains an open problem. To this end, we leverage a novel distributed machine learning paradigm, i.e. , federated learning, to realize effective on-device SNN training. As current FL schemes often train only one model with fixed architecture, and the existing SNN training algorithm is resource-intensive, integrating FL and SNN is non-trivial. Furthermore, two intrinsic features in edge computing, i.e. , data and system heterogeneity, exacerbate the difficulty. Motivated by this, we redesign the model distribution, local training, and model aggregation phases in traditional FL, and propose FedSNN, a framework that ensures all widths in SNN can obtain high accuracy with less resource consumption. Specifically, for devices with heterogeneous training capacities and data distributions, the parameter server will distribute each of them with one proper width for adaptive local training guided by their uploaded model features, and their trained models will be weighted-averaged using the proposed multi-width SNN aggregation to improve their statistical utility. Extensive experiments on a distributed testbed show that FedSNN improves the model accuracy by about 2.18%-8.1%, and accelerates training by about 1.31 $\times$ -6.84 $\times$ , compared with existing solutions.

The Robustness of Spiking Neural Networks in Communication and its Application towards Network Efficiency in Federated Learning

Communication Trade-offs in Federated Learning of Spiking Neural Networks

Message Passing Based Wireless Federated Learning Via Analog Message Aggregation

Efficient Asynchronous Federated Neuromorphic Learning of Spiking Neural Networks.

Federated Neuromorphic Learning of Spiking Neural Networks for Low-Power Edge Intelligence

Efficient Model Compression via Global Sparsification for Over-the-Air Federated Learning.

Asynchronous Federated Learning over Wireless Communication Networks.

Communication-Efficient Federated Learning with Binary Neural Networks

Communication-Efficient Federated Learning: A Variance-Reduced Stochastic Approach with Adaptive Sparsification.

A Communication-Efficient Model of Sparse Neural Network for Distributed Intelligence

Federated Learning for Spiking Neural Networks by Hint-Layer Knowledge Distillation

Accelerating Split Federated Learning over Wireless Communication Networks

Spiking Neural Networks in Vertical Federated Learning: Performance Trade-offs

SNN-SC: A Spiking Semantic Communication Framework for Collaborative Intelligence

FedSNN: Training Slimmable Neural Network with Federated Learning in Edge Computing

Spiking Neural Networks -- Part III: Neuromorphic Communications

Communication-Efficient Federated Learning via Regularized Sparse Random Networks

Heterogeneous Federated Learning with Convolutional and Spiking Neural Networks

DynSNN: A Dynamic Approach to Reduce Redundancy in Spiking Neural Networks

Recent Advances in Scalable Energy-Efficient and Trustworthy Spiking Neural networks: from Algorithms to Technology