sBSNN: Stochastic-Bits Enabled Binary Spiking Neural Network with On-Chip Learning for Energy Efficient Neuromorphic Computing at the Edge

Minsuk Koo,Gopalakrishnan Srinivasan,Yong Shim,Kaushik Roy
DOI: https://doi.org/10.48550/arXiv.2002.11163
2020-02-26
Abstract:In this work, we propose stochastic Binary Spiking Neural Network (sBSNN) composed of stochastic spiking neurons and binary synapses (stochastic only during training) that computes probabilistically with one-bit precision for power-efficient and memory-compressed neuromorphic computing. We present an energy-efficient implementation of the proposed sBSNN using 'stochastic bit' as the core computational primitive to realize the stochastic neurons and synapses, which are fabricated in 90nm CMOS process, to achieve efficient on-chip training and inference for image recognition tasks. The measured data shows that the 'stochastic bit' can be programmed to mimic spiking neurons, and stochastic Spike Timing Dependent Plasticity (or sSTDP) rule for training the binary synaptic weights without expensive random number generators. Our results indicate that the proposed sBSNN realization offers possibility of up to 32x neuronal and synaptic memory compression compared to full precision (32-bit) SNN and energy efficiency of 89.49 TOPS/Watt for two-layer fully-connected SNN.
Emerging Technologies,Hardware Architecture
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to implement a high - performance and low - power neuromorphic computing system on edge devices. Specifically, the paper proposes a binary spiking neural network based on "stochastic bit" (sBSNN), aiming to achieve energy - and memory - efficient neuromorphic computing through probabilistic computing with 1 - bit precision. This network is especially suitable for resource - constrained edge devices, such as drones, self - driving cars, and smart wearable devices, etc. These devices need to process a large amount of data in real - time and have strict requirements for energy consumption. ### Main Problems and Solutions 1. **Problem: Energy Efficiency and Computing Power** - **Background**: With the popularization of autonomous intelligent systems, edge devices need to process a large amount of data, which poses higher requirements for computing power and energy efficiency. The application of traditional neural networks on these devices is limited because they usually require high computing resources and energy consumption. - **Solution**: The paper proposes a new neural network architecture - sBSNN, which uses "stochastic bit" as the core computing unit to achieve probabilistic computing. This design not only reduces the energy required for computing but also significantly compresses the storage requirements of neurons and synapses. 2. **Problem: Real - Time Online Learning** - **Background**: Edge devices need to have the ability of real - time learning in order to extract useful information from real - time inputs. Traditional offline training methods not only have high latency but also pose security risks. - **Solution**: sBSNN supports on - chip online learning and uses the stochastic spiking - time - dependent plasticity (sSTDP) algorithm to train binary synaptic weights. This learning rule is not only simple and efficient but can also be implemented without an expensive random number generator. 3. **Problem: Hardware Implementation** - **Background**: Existing neural network hardware implementations usually require complex circuit designs, resulting in high energy consumption and cost. - **Solution**: The paper elaborates on how to implement the hardware design of sBSNN in a 90 - nm CMOS process. By using "stochastic bit" as the core computing unit, this design not only simplifies the circuit structure but also significantly improves energy efficiency. Experimental results show that the energy efficiency of sBSNN reaches 89.49 TOPS/Watt, much higher than that of traditional full - precision SNNs. ### Key Technologies 1. **"Stochastic bit"** - **Function**: As the core computing unit of sBSNN, it can switch between logic low and high states in a probabilistic manner, and its switching probability changes in an S - shaped curve. - **Implementation**: It is implemented by cross - coupled inverters and PMOS/NMOS transistors, and has the characteristics of low leakage power and high energy efficiency. 2. **Stochastic spiking - time - dependent plasticity (sSTDP)** - **Function**: It is used to train binary synaptic weights and adjusts the probability of synaptic weights according to the time difference between forward and backward spikes. - **Formula**: \[ P_{L \rightarrow H} = \gamma_{\text{pot}} \cdot e^{-\Delta t / \tau_{\text{pot}}} \quad \text{when} \Delta t = t_{\text{post}} - t_{\text{pre}} > 0 \] \[ P_{H \rightarrow L} = \gamma_{\text{dep}} \cdot e^{\Delta t / \tau_{\text{dep}}} \quad \text{when} \Delta t = t_{\text{post}} - t_{\text{pre}} < 0 \] - **Explanation**: Here, \( P_{L \rightarrow H} \) and \( P_{H \rightarrow L} \) represent the probabilities of potentiation and depression respectively, and \( \gamma_{\text{pot}} \) and \( \gamma_{\text{dep}} \) represent...