Abstract:In recent years the field of neuromorphic low-power systems that consume orders of magnitude less power gained significant momentum. However, their wider use is still hindered by the lack of algorithms that can harness the strengths of such architectures. While neuromorphic adaptations of representation learning algorithms are now emerging, efficient processing of temporal sequences or variable length-inputs remain difficult. Recurrent neural networks (RNN) are widely used in machine learning to solve a variety of sequence learning tasks. In this work we present a train-and-constrain methodology that enables the mapping of machine learned (Elman) RNNs on a substrate of spiking neurons, while being compatible with the capabilities of current and near-future neuromorphic systems. This "train-and-constrain" method consists of first training RNNs using backpropagation through time, then discretizing the weights and finally converting them to spiking RNNs by matching the responses of artificial neurons with those of the spiking neurons. We demonstrate our approach by mapping a natural language processing task (question classification), where we demonstrate the entire mapping process of the recurrent layer of the network on IBM's Neurosynaptic System "TrueNorth", a spike-based digital neuromorphic hardware architecture. TrueNorth imposes specific constraints on connectivity, neural and synaptic parameters. To satisfy these constraints, it was necessary to discretize the synaptic weights and neural activities to 16 levels, and to limit fan-in to 64 inputs. We find that short synaptic delays are sufficient to implement the dynamical (temporal) aspect of the RNN in the question classification task. The hardware-constrained model achieved 74% accuracy in question classification while using less than 0.025% of the cores on one TrueNorth chip, resulting in an estimated power consumption of ~17 uW.

Improving energy efficiency and classification accuracy of neuromorphic chips by learning binary synaptic crossbars

Improving classification accuracy of feedforward neural networks for spiking neuromorphic chips

Exploiting Inherent Error-Resiliency of Neuromorphic Computing to achieve Extreme Energy-Efficiency through Mixed-Signal Neurons

NBSSN: A Neuromorphic Binary Single-Spike Neural Network for Efficient Edge Intelligence.

Binary Resistive-Switching-Device-Based Electronic Synapse with Spike-Rate-Dependent Plasticity for Online Learning

sBSNN: Stochastic-Bits Enabled Binary Spiking Neural Network with On-Chip Learning for Energy Efficient Neuromorphic Computing at the Edge

Energy-efficient neuromorphic classifiers

Binary stochasticity enabled highly efficient neuromorphic deep learning achieves better-than-software accuracy

Power-efficient neural network with artificial dendrites

You Only Spike Once: Improving Energy-Efficient Neuromorphic Inference to ANN-Level Accuracy

55 Nm CMOS Mixed-Signal Neuromorphic Circuits for Constructing Energy-Efficient Reconfigurable SNNs

Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardware

A Scatter-and-Gather Spiking Convolutional Neural Network on a Reconfigurable Neuromorphic Hardware

An Efficient Neuromorphic Implementation of Temporal Coding-Based On-Chip STDP Learning

Fast and Accurate Sparse Coding of Visual Stimuli With a Simple, Ultralow-Energy Spiking Architecture

A Spiking Neuromorphic Design with Resistive Crossbar

Sparsity-Aware In-Memory Neuromorphic Computing Unit With Configurable Topology of Hybrid Spiking and Artificial Neural Network

Ultralow Power Consumption and Large Dynamic Range Synaptic Transistor Based on Α-In2se3 Nanosheets

An error-propagation spiking neural network compatible with neuromorphic processors

The Backpropagation Algorithm Implemented on Spiking Neuromorphic Hardware

An Energy-Efficient Computing-in-Memory Neuromorphic System with On-Chip Training.