Abstract:While neuromorphic systems may be the ultimate platform for deploying spiking neural networks (SNNs), their distributed nature and optimization for specific types of models makes them unwieldy tools for developing them. Instead, SNN models tend to be developed and simulated on computers or clusters of computers with standard von Neumann CPU architectures. Over the last decade, as well as becoming a common fixture in many workstations, NVIDIA GPU accelerators have entered the High Performance Computing field and are now used in 50 % of the Top 10 super computing sites worldwide. In this paper we use our GeNN code generator to re-implement two neo-cortex-inspired, circuit-scale, point neuron network models on GPU hardware. We verify the correctness of our GPU simulations against prior results obtained with NEST running on traditional HPC hardware and compare the performance with respect to speed and energy consumption against published data from CPU-based HPC and neuromorphic hardware. A full-scale model of a cortical column can be simulated at speeds approaching 0.5× real-time using a single NVIDIA Tesla V100 accelerator-faster than is currently possible using a CPU based cluster or the SpiNNaker neuromorphic system. In addition, we find that, across a range of GPU systems, the energy to solution as well as the energy per synaptic event of the microcircuit simulation is as much as 14× lower than either on SpiNNaker or in CPU-based simulations. Besides performance in terms of speed and energy consumption of the simulation, efficient initialization of models is also a crucial concern, particularly in a research context where repeated runs and parameter-space exploration are required. Therefore, we also introduce in this paper some of the novel parallel initialization methods implemented in the latest version of GeNN and demonstrate how they can enable further speed and energy advantages.

Real-Time Cortical Simulation on Neuromorphic Hardware

A System-on-Chip Based Hybrid Neuromorphic Compute Node Architecture for Reproducible Hyper-Real-Time Simulations of Spiking Neural Networks

SpiNNaker: A 1-W 18-Core System-on-Chip for Massively-Parallel Neural Network Simulation

SpikeNC: an Accurate and Scalable Simulator for Spiking Neural Network on Multi-Core Neuromorphic Hardware

SuperNeuro: A Fast and Scalable Simulator for Neuromorphic Computing

Algorithm and Software for Simulation of Spiking Neural Networks on the Multi-Chip SpiNNaker System

GPUs Outperform Current HPC and Neuromorphic Solutions in Terms of Speed and Energy When Simulating a Highly-Connected Cortical Model

Real-Time Neuromorphic System for Large-Scale Conductance-Based Spiking Neural Networks.

Algorithms for Fast Spiking Neural Network Simulation on FPGAs

DYNAP-SE2: a scalable multi-core dynamic neuromorphic asynchronous spiking neural network processor

Scalable Event-Driven Native Parallel Processing

Sub-realtime simulation of a neuronal network of natural density

Mapping Very Large Scale Spiking Neuron Network to Neuromorphic Hardware.

Scaling of a large-scale simulation of synchronous slow-wave and asynchronous awake-like activity of a cortical model with long-range interconnections

Managing Burstiness and Scalability in Event-Driven Models on the SpiNNaker Neuromimetic System.

Efficient Modelling of Spiking Neural Networks on a Scalable Chip Multiprocessor

Fast simulations of highly-connected spiking cortical models using GPUs

Extensible neuromorphic computing simulator based on a programmable hardware

Demonstrating the Advantages of Analog Wafer-Scale Neuromorphic Hardware

Performance Evaluation of Spintronic-Based Spiking Neural Networks Using Parallel Discrete-Event Simulation

sPyNNaker: A Software Package for Running PyNN Simulations on SpiNNaker