Abstract:In recent years the field of neuromorphic low-power systems that consume orders of magnitude less power gained significant momentum. However, their wider use is still hindered by the lack of algorithms that can harness the strengths of such architectures. While neuromorphic adaptations of representation learning algorithms are now emerging, efficient processing of temporal sequences or variable length-inputs remain difficult. Recurrent neural networks (RNN) are widely used in machine learning to solve a variety of sequence learning tasks. In this work we present a train-and-constrain methodology that enables the mapping of machine learned (Elman) RNNs on a substrate of spiking neurons, while being compatible with the capabilities of current and near-future neuromorphic systems. This "train-and-constrain" method consists of first training RNNs using backpropagation through time, then discretizing the weights and finally converting them to spiking RNNs by matching the responses of artificial neurons with those of the spiking neurons. We demonstrate our approach by mapping a natural language processing task (question classification), where we demonstrate the entire mapping process of the recurrent layer of the network on IBM's Neurosynaptic System "TrueNorth", a spike-based digital neuromorphic hardware architecture. TrueNorth imposes specific constraints on connectivity, neural and synaptic parameters. To satisfy these constraints, it was necessary to discretize the synaptic weights and neural activities to 16 levels, and to limit fan-in to 64 inputs. We find that short synaptic delays are sufficient to implement the dynamical (temporal) aspect of the RNN in the question classification task. The hardware-constrained model achieved 74% accuracy in question classification while using less than 0.025% of the cores on one TrueNorth chip, resulting in an estimated power consumption of ~17 uW.

Supervised training of spiking neural networks for robust deployment on mixed-signal neuromorphic processors

Improving classification accuracy of feedforward neural networks for spiking neuromorphic chips

Neuromorphic analog circuits for robust on-chip always-on learning in spiking neural networks

Experimental Demonstration of Supervised Learning in Spiking Neural Networks with Phase-Change Memory Synapses

An error-propagation spiking neural network compatible with neuromorphic processors

Hardware calibrated learning to compensate heterogeneity in analog RRAM-based Spiking Neural Networks

Conversion of Artificial Recurrent Neural Networks to Spiking Neural Networks for Low-power Neuromorphic Hardware

Direct Training of Hardware-Friendly Weight Binarized Spiking Neural Network with Surrogate Gradient Learning Towards Spatio-Temporal Event-Based Dynamic Data Recognition

Gradient-descent hardware-aware training and deployment for mixed-signal Neuromorphic processors

Implementing Efficient Balanced Networks with Mixed-Signal Spike-Based Learning Circuits

In-situ Learning in Hardware Compatible Multi-layer Memristive Spiking Neural Network

Recent Advances in Scalable Energy-Efficient and Trustworthy Spiking Neural networks: from Algorithms to Technology

Enhancing Robustness of Memristor Crossbar‐Based Spiking Neural Networks against Nonidealities: A Hybrid Approach for Neuromorphic Computing in Noisy Environments

Sparsity-Aware In-Memory Neuromorphic Computing Unit With Configurable Topology of Hybrid Spiking and Artificial Neural Network

PC-SNN: Supervised Learning with Local Hebbian Synaptic Plasticity Based on Predictive Coding in Spiking Neural Networks

Spiking recurrent neural networks for neuromorphic computing in nonlinear structural mechanics

In-Hardware Learning of Multilayer Spiking Neural Networks on a Neuromorphic Processor

Unsupervised Learning on Resistive Memory Array Based Spiking Neural Networks

Towards Efficient Deployment of Hybrid SNNs on Neuromorphic and Edge AI Hardware

Efficient Implementation of Spiking Neural Networks for Inference Using Ex-Situ Training

A A 22nm 0.43pJ/SOP Sparsity-Aware In-Memory Neuromorphic Computing System with Hybrid Spiking and Artificial Neural Network and Configurable Topology