Abstract:Artificial neural networks (ANNs), like convolutional neural networks (CNNs), have achieved the state-of-the-art results for many machine learning tasks. However, inference with large-scale full-precision CNNs must cause substantial energy consumption and memory occupation, which seriously hinders their deployment on mobile and embedded systems. Highly inspired from biological brain, spiking neural networks (SNNs) are emerging as new solutions because of natural superiority in brain-like learning and great energy efficiency with event-driven communication and computation. Nevertheless, training a deep SNN remains a main challenge and there is usually a big accuracy gap between ANNs and SNNs. In this paper, we introduce a hardware-friendly conversion algorithm called “scatter-and-gather” to convert quantized ANNs to lossless SNNs, where neurons are connected with ternary {−1,0,1} synaptic weights. Each spiking neuron is stateless and more like original McCulloch and Pitts model, because it fires at most one spike and need be reset at each time step. Furthermore, we develop an incremental mapping framework to demonstrate efficient network deployments on a reconfigurable neuromorphic chip. Experimental results show our spiking LeNet on MNIST and VGG-Net on CIFAR-10 datasetobtain 99.37% and 91.91% classification accuracy, respectively. Besides, the presented mapping algorithm manages network deployment on our neuromorphic chip with maximum resource efficiency and excellent flexibility. Our four-spike LeNet and VGG-Net on chip can achieve respective real-time inference speed of 0.38 ms/image, 3.24 ms/image, and an average power consumption of 0.28 mJ/image and 2.3 mJ/image at 0.9 V, 252 MHz, which is nearly two orders of magnitude more efficient than traditional GPUs.

Artificial Neural Network Computation On Graphic Process Unit

Self-Organizing Maps Computing on Graphic Process Unit

Large Scale Recurrent Neural Network on GPU

Large Scale Artificial Neural Network Training Using Multi-GPUs

Implementation and evaluation of deep neural networks (DNN) on mainstream heterogeneous systems

GPU-accelerated artificial neural network potential for molecular dynamics simulation

A novel hardware accelerator guideline for ANN with high performance

A parallel computing platform for training large scale neural networks

Genetic Algorithm Supported by Graphical Processing Unit Improves the Exploration of Effective Connectivity in Functional Brain Imaging.

High Performance Simulation of Spiking Neural Network on GPGPUs

GPU-utilized Composite Quantization for Large-scale High-dimension ANN Search

Towards Efficient Neural Networks On-a-chip: Joint Hardware-Algorithm Approaches

A Scatter-and-Gather Spiking Convolutional Neural Network on a Reconfigurable Neuromorphic Hardware

A parallel implementation of nearest neighbor analysis based on GPGPU

A Power Efficient Neural Network Implementation on Heterogeneous FPGA and GPU Devices

Special Topic on Nonvolatile Memory for Efficient Implementation of Neural/Neuromorphic Computing

Neural Network Transformation Under Hardware Constraints

Making Human Connectome Faster: GPU Acceleration of Brain Network Analysis

GPU-accelerated Proximity Graph Approximate Nearest Neighbor Search and Construction

NPGPU: Network processing on graphics processing units

Implementation of K-nearest Neighbor Algorithm Based on GPU