Abstract:With large deep neural networks (DNNs) necessary to solve complex and data-intensive problems, energy efficiency is a key bottleneck for effectively deploying DL in the real world. Deep spiking NNs have gained much research attention recently due to the interest in building biological neural networks and the availability of neuromorphic platforms, which can be orders of magnitude more energy efficient compared to CPUs and GPUs. Although spiking NNs have proven to be an efficient technique for solving many machine learning and computer vision problems, to the best of our knowledge, this is the first attempt to adapt spiking NNs to sparse datasets. In this paper, we study the behaviour of spiking NNs in handling NLP datasets and the sparsity in their data representation. Then, we propose a novel framework for spiking NN using the concept of stochastic computing. Specifically, instead of generating spike trains with firing rates proportional to the intensity of each value in the feature set separately, the whole feature set is treated as a distribution function and a stochastic spiking train that follow this distribution is generated. This framework reduces the connectivity between NN layers from $\mathrm{O}(N)$ to $\mathrm{O}(\log N)$. Also, it encodes input data differently and make suitable to handle sparse datasets. Finally, the framework achieves high energy efficiency since it uses Integrate and Fire neurons same as conventional spiking NNs. The results show that our proposed stochastic-based SNN achieves nearly the same accuracy as the original DNN on MNIST dataset, and it has better performance than state-of-the-art SNN. Besides that stochastic-based SNN is energy efficient, where the fully connected DNN, the conventional SNN, and the data normalized SNN consume 38.24, 1.83, and 1.85-times more energy than the stochastic-based SNN, respectively. For sparse datasets, including IMDb and In-House clinical datasets, stochastic-based SNN achieves performance comparable to that of the conventional DNN. However, the conventional spiking NN has a significant decline in classification accuracy.

Efficient Asynchronous Federated Neuromorphic Learning of Spiking Neural Networks.

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

The Robustness of Spiking Neural Networks in Communication and its Application towards Network Efficiency in Federated Learning

Communication Trade-offs in Federated Learning of Spiking Neural Networks

Heterogeneous Federated Learning with Convolutional and Spiking Neural Networks

Asynchronous Federated Learning over Wireless Communication Networks.

Spiking Deep Residual Networks.

Efficient Spiking Neural Networks with Sparse Selective Activation for Continual Learning

Federated Learning for Spiking Neural Networks by Hint-Layer Knowledge Distillation

Spiking Neural Networks in Vertical Federated Learning: Performance Trade-offs

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

FedSNN: Training Slimmable Neural Network with Federated Learning in Edge Computing

Comprehensive Online Training and Deployment for Spiking Neural Networks

Joint A-SNN: Joint Training of Artificial and Spiking Neural Networks via Self-Distillation and Weight Factorization

Recent Advances in Scalable Energy-Efficient and Trustworthy Spiking Neural networks: from Algorithms to Technology

A A 22nm 0.43pJ/SOP Sparsity-Aware In-Memory Neuromorphic Computing System with Hybrid Spiking and Artificial Neural Network and Configurable Topology

Towards Scalable GPU-Accelerated SNN Training via Temporal Fusion

Fast and Efficient Asynchronous Neural Computation with Adapting Spiking Neural Networks

Energy efficient stochastic-based deep spiking neural networks for sparse datasets

Training Spiking Neural Networks with Accumulated Spiking Flow.