Abstract:Simulation of large scale biologically plausible spiking neural networks, e.g., Bayesian Confidence Propagation Neural Network (BCPNN), usually requires high-performance supercomputers with dedicated accelerators, such as GPUs, FPGAs, or even Application-Specific Integrated Circuits (ASICs). Almost all of these computers are based on the von Neumann architecture that separates storage and computation. In all these solutions, memory access is the dominant cost even for highly customized computation and memory architecture, such as ASICs. In this paper, we propose an optimization technique that can make the BCPNN simulation memory access friendly by avoiding a dual-access pattern. The BCPNN synaptic traces and weights are organized as matrices accessed both row-wise and column-wise. Accessing data stored in DRAM with a dual-access pattern is extremely expensive. A post-synaptic history buffer and an approximation function thus are introduced to eliminate the troublesome column update. The error analysis combining theoretical analysis and experiments suggests that the probability of introducing intolerable errors by such optimization can be bounded to a very small number, which makes it almost negligible. Derivation and validation of such a bound is the core contribution of this paper. Experiments on a GPU platform shows that compared to the previously reported baseline simulation strategy, the proposed optimization technique reduces the storage requirement by 33%, the global memory access demand by more than 27% and DRAM access rate by more than 5%; the latency of updating synaptic traces decreases by roughly 50%. Compared with the other similar optimization technique reported in the literature, our method clearly shows considerably better results. Although the BCPNN is used as the targeted neural network model, the proposed optimization method can be applied to other artificial neural network models based on a Hebbian learning rule.

Memory-efficient neurons and synapses for spike-timing-dependent-plasticity in large-scale spiking networks

Memory-Efficient Synaptic Connectivity for Spike-Timing- Dependent Plasticity

Toward Efficient Processing and Learning with Spikes: New Approaches for Multispike Learning

A Spike-Based Model of Neuronal Intrinsic Plasticity.

Precise-Spike-Driven Synaptic Plasticity: Learning Hetero-Association of Spatiotemporal Spike Patterns

Learning of Precise Spike Times with Membrane Potential Dependent Synaptic Plasticity

A framework for studying synaptic plasticity with neural spike train data

Improving Multispike Learning with Plastic Synaptic Delays

Time-Integrated Spike-Timing-Dependent-Plasticity

A simple model for Behavioral Time Scale Synaptic Plasticity provides content addressable memory with binary synapses and one-shot learning

How synaptic plasticity influences spike synchronization and its transitions in complex neuronal network.

Extending dynamic memory of spiking neuron networks

Forward Table-Based Presynaptic Event-Triggered Spike-Timing-Dependent Plasticity

Learning arbitrary dynamics in efficient, balanced spiking networks using local plasticity rules

Optimizing BCPNN Learning Rule for Memory Access

Optimality of short-term synaptic plasticity in modelling certain dynamic environments

A system of interacting neurons with short term synaptic facilitation

A Spike-Timing Pattern Based Neural Network Model for the Study of Memory Dynamics

Spike Timing-Dependent Plasticity in CA1 Pyramidal Neuron-Controlling Hippocampal Circuits: a Model Study

Prominent characteristics of recurrent neuronal networks are robust against low synaptic weight resolution

Implementation of Memristive Neural Networks with Spike-Rate-dependent Plasticity Synapses