Abstract:With large deep neural networks (DNNs) necessary to solve complex and data-intensive problems, energy efficiency is a key bottleneck for effectively deploying DL in the real world. Deep spiking NNs have gained much research attention recently due to the interest in building biological neural networks and the availability of neuromorphic platforms, which can be orders of magnitude more energy efficient compared to CPUs and GPUs. Although spiking NNs have proven to be an efficient technique for solving many machine learning and computer vision problems, to the best of our knowledge, this is the first attempt to adapt spiking NNs to sparse datasets. In this paper, we study the behaviour of spiking NNs in handling NLP datasets and the sparsity in their data representation. Then, we propose a novel framework for spiking NN using the concept of stochastic computing. Specifically, instead of generating spike trains with firing rates proportional to the intensity of each value in the feature set separately, the whole feature set is treated as a distribution function and a stochastic spiking train that follow this distribution is generated. This framework reduces the connectivity between NN layers from $\mathrm{O}(N)$ to $\mathrm{O}(\log N)$. Also, it encodes input data differently and make suitable to handle sparse datasets. Finally, the framework achieves high energy efficiency since it uses Integrate and Fire neurons same as conventional spiking NNs. The results show that our proposed stochastic-based SNN achieves nearly the same accuracy as the original DNN on MNIST dataset, and it has better performance than state-of-the-art SNN. Besides that stochastic-based SNN is energy efficient, where the fully connected DNN, the conventional SNN, and the data normalized SNN consume 38.24, 1.83, and 1.85-times more energy than the stochastic-based SNN, respectively. For sparse datasets, including IMDb and In-House clinical datasets, stochastic-based SNN achieves performance comparable to that of the conventional DNN. However, the conventional spiking NN has a significant decline in classification accuracy.

A Sparse Event-Driven Unsupervised Learning Network with Adaptive Exponential Integrate-and-Fire Model

RSNN: Recurrent Spiking Neural Networks for Dynamic Spatial-Temporal Information Processing

Deep CovDenseSNN: A Hierarchical Event-Driven Dynamic Framework with Spiking Neurons in Noisy Environment

Efficient Spiking Neural Networks with Sparse Selective Activation for Continual Learning

Adaptive Multi-Level Firing for Direct Training Deep Spiking Neural Networks

Event-driven Spiking Neural Networks with Spike-Based Learning

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

An Energy Efficient STDP-Based SNN Architecture With On-Chip Learning

High-parallelism Inception-like Spiking Neural Networks for Unsupervised Feature Learning

Spiking Deep Residual Networks.

A A 22nm 0.43pJ/SOP Sparsity-Aware In-Memory Neuromorphic Computing System with Hybrid Spiking and Artificial Neural Network and Configurable Topology

Fast-Learning and High-Robustness Unsupervised Spiking Neural Networks with a Highly Parallel Inception-like Architecture

30.2 A 22nm 0.26nW/Synapse Spike-Driven Spiking Neural Network Processing Unit Using Time-Step-First Dataflow and Sparsity-Adaptive In-Memory Computing

Randomize and Match: Exploiting Irregular Sparsity for Energy Efficient Processing in SNNs

Energy efficient stochastic-based deep spiking neural networks for sparse datasets

A Scatter-and-Gather Spiking Convolutional Neural Network on a Reconfigurable Neuromorphic Hardware

Accelerating SNN Training with Stochastic Parallelizable Spiking Neurons

Event-Driven Learning for Spiking Neural Networks

Reconstruction of Adaptive Leaky Integrate-and-Fire Neuron to Enhance the Spiking Neural Networks Performance by Establishing Complex Dynamics

FSpiNN: An Optimization Framework for Memory- and Energy-Efficient Spiking Neural Networks

Accurate and Efficient Event-based Semantic Segmentation Using Adaptive Spiking Encoder-Decoder Network