Abstract:Spiking neural networks (SNNs) are a computational tool in which the information is coded into spikes, as in some parts of the brain, differently from conventional neural networks (NNs) that compute over real-numbers. Therefore, SNNs can implement intelligent information extraction in real-time at the edge of data acquisition and correspond to a complementary solution to conventional NNs working for cloud-computing. Both NN classes face hardware constraints due to limited computing parallelism and separation of logic and memory. Emerging memory devices, like resistive switching memories, phase change memories, or memristive devices in general are strong candidates to remove these hurdles for NN applications. The well-established training procedures of conventional NNs helped in defining the desiderata for memristive device dynamics implementing synaptic units. The generally agreed requirements are a linear evolution of memristive conductance upon stimulation with train of identical pulses and a symmetric conductance change for conductance increase and decrease. Conversely, little work has been done to understand the main properties of memristive devices supporting efficient SNN operation. The reason lies in the lack of a background theory for their training. As a consequence, requirements for NNs have been taken as a reference to develop memristive devices for SNNs. In the present work, we show that, for efficient CMOS/memristive SNNs, the requirements for synaptic memristive dynamics are very different from the needs of a conventional NN. System-level simulations of a SNN trained to classify hand-written digit images through a spike timing dependent plasticity protocol are performed considering various linear and non-linear plausible synaptic memristive dynamics. We consider memristive dynamics bounded by artificial hard conductance values and limited by the natural dynamics evolution toward asymptotic values (soft-boundaries). We quantitatively analyze the impact of resolution and non-linearity properties of the synapses on the network training and classification performance. Finally, we demonstrate that the non-linear synapses with hard boundary values enable higher classification performance and realize the best trade-off between classification accuracy and required training time. With reference to the obtained results, we discuss how memristive devices with non-linear dynamics constitute a technologically convenient solution for the development of on-line SNN training.

Parallel Synapses with Transmission Nonlinearities Enhance Neuronal Classification Capacity

Impact of dendritic non-linearities on the computational capabilities of neurons

Toward Efficient Processing and Learning with Spikes: New Approaches for Multispike Learning

Parallel Spiking Neurons with High Efficiency and Ability to Learn Long-term Dependencies

Efficient supervised learning in networks with binary synapses

Multibranch Formal Neuron: An Internally Nonlinear Learning Unit

A Brain-Inspired Spiking Neural Network Model with Temporal Encoding and Learning

SynapsNet: Enhancing Neuronal Population Dynamics Modeling via Learning Functional Connectivity

Hierarchical Spiking-Based Model for Efficient Image Classification with Enhanced Feature Extraction and Encoding.

Non-linear Memristive Synaptic Dynamics for Efficient Unsupervised Learning in Spiking Neural Networks

High-parallelism Inception-like Spiking Neural Networks for Unsupervised Feature Learning

Can Single Neurons Solve MNIST? The Computational Power of Biological Dendritic Trees

The Role of Spiking Nonlinearity in Contrast Gain Control and Information Transmission

Mitigating Communication Costs in Neural Networks: The Role of Dendritic Nonlinearity

Fast-Learning and High-Robustness Unsupervised Spiking Neural Networks with a Highly Parallel Inception-like Architecture

On the Benefits of Biophysical Synapses

Improving energy efficiency and classification accuracy of neuromorphic chips by learning binary synaptic crossbars

Turn Down that Noise: Synaptic Encoding of Afferent SNR in a Single Spiking Neuron

A spiking photonic neural network of 40.000 neurons, trained with rank-order coding for leveraging sparsity

Few-shot pattern detection by transient boosting of somato-dendritic coupling