Abstract:While neuromorphic systems may be the ultimate platform for deploying spiking neural networks (SNNs), their distributed nature and optimization for specific types of models makes them unwieldy tools for developing them. Instead, SNN models tend to be developed and simulated on computers or clusters of computers with standard von Neumann CPU architectures. Over the last decade, as well as becoming a common fixture in many workstations, NVIDIA GPU accelerators have entered the High Performance Computing field and are now used in 50 % of the Top 10 super computing sites worldwide. In this paper we use our GeNN code generator to re-implement two neo-cortex-inspired, circuit-scale, point neuron network models on GPU hardware. We verify the correctness of our GPU simulations against prior results obtained with NEST running on traditional HPC hardware and compare the performance with respect to speed and energy consumption against published data from CPU-based HPC and neuromorphic hardware. A full-scale model of a cortical column can be simulated at speeds approaching 0.5× real-time using a single NVIDIA Tesla V100 accelerator-faster than is currently possible using a CPU based cluster or the SpiNNaker neuromorphic system. In addition, we find that, across a range of GPU systems, the energy to solution as well as the energy per synaptic event of the microcircuit simulation is as much as 14× lower than either on SpiNNaker or in CPU-based simulations. Besides performance in terms of speed and energy consumption of the simulation, efficient initialization of models is also a crucial concern, particularly in a research context where repeated runs and parameter-space exploration are required. Therefore, we also introduce in this paper some of the novel parallel initialization methods implemented in the latest version of GeNN and demonstrate how they can enable further speed and energy advantages.

mlGeNN: accelerating SNN inference using GPU-enabled neural networks

Brian2GeNN: accelerating spiking neural network simulations with graphics hardware

SpikeConverter: an Efficient Conversion Framework Zipping the Gap Between Artificial Neural Networks and Spiking Neural Networks

Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networks

A Scatter-and-Gather Spiking Convolutional Neural Network on a Reconfigurable Neuromorphic Hardware

Optimal ANN-SNN Conversion with Group Neurons

Fast-SNN: Fast Spiking Neural Network by Converting Quantized ANN

Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation

An Efficient Learning Algorithm for Direct Training Deep Spiking Neural Networks

Optimal ANN-SNN Conversion for High-accuracy and Ultra-low-latency Spiking Neural Networks

BSNN: Towards faster and better conversion of artificial neural networks to spiking neural networks with bistable neurons

Spike Trains Encoding and Threshold Rescaling Method for Deep Spiking Neural Networks

Constructing Lightweight and Efficient Spiking Neural Networks for EEG-based Motor Imagery Classification

Towards Scalable GPU-Accelerated SNN Training via Temporal Fusion

A universal ANN-to-SNN framework for achieving high accuracy and low latency deep Spiking Neural Networks

Bridging the Gap between ANNs and SNNs by Calibrating Offset Spikes

Efficient Converted Spiking Neural Network for 3D and 2D Classification

Training-free Conversion of Pretrained ANNs to SNNs for Low-Power and High-Performance Applications

GPUs Outperform Current HPC and Neuromorphic Solutions in Terms of Speed and Energy When Simulating a Highly-Connected Cortical Model

Efficient and Accurate Conversion of Spiking Neural Network with Burst Spikes

Optimized Potential Initialization for Low-latency Spiking Neural Networks