Abstract:Simulation of large scale biologically plausible spiking neural networks, e.g., Bayesian Confidence Propagation Neural Network (BCPNN), usually requires high-performance supercomputers with dedicated accelerators, such as GPUs, FPGAs, or even Application-Specific Integrated Circuits (ASICs). Almost all of these computers are based on the von Neumann architecture that separates storage and computation. In all these solutions, memory access is the dominant cost even for highly customized computation and memory architecture, such as ASICs. In this paper, we propose an optimization technique that can make the BCPNN simulation memory access friendly by avoiding a dual-access pattern. The BCPNN synaptic traces and weights are organized as matrices accessed both row-wise and column-wise. Accessing data stored in DRAM with a dual-access pattern is extremely expensive. A post-synaptic history buffer and an approximation function thus are introduced to eliminate the troublesome column update. The error analysis combining theoretical analysis and experiments suggests that the probability of introducing intolerable errors by such optimization can be bounded to a very small number, which makes it almost negligible. Derivation and validation of such a bound is the core contribution of this paper. Experiments on a GPU platform shows that compared to the previously reported baseline simulation strategy, the proposed optimization technique reduces the storage requirement by 33%, the global memory access demand by more than 27% and DRAM access rate by more than 5%; the latency of updating synaptic traces decreases by roughly 50%. Compared with the other similar optimization technique reported in the literature, our method clearly shows considerably better results. Although the BCPNN is used as the targeted neural network model, the proposed optimization method can be applied to other artificial neural network models based on a Hebbian learning rule.

FPGA-Based HPC for Associative Memory System

Scalable Multi-FPGA HPC Architecture for Associative Memory System

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

A FPGA-based Hardware Accelerator for Bayesian Confidence Propagation Neural Network

FeFET-Based In-Memory Hyperdimensional Encoding Design

A Hardware Implementation of SNN-Based Spatio-Temporal Memory Model.

FPGA implementation of hardware processing modules as coprocessors in brain-machine interfaces.

A FeFET-based Time-Domain Associative Memory for Multi-bit Similarity Computation

FPGA-Based Near-Memory Acceleration of Modern Data-Intensive Applications

Heterogeneous Systems with Reconfigurable Neuromorphic Computing Accelerators

A heterogeneous computing system with memristor-based neuromorphic accelerators

Parallel FPGA-based All Pairs Shortest Paths for Sparse Networks: A Human Brain Connectome Case Study.

Programmable FPGA-based Memory Controller

Accelerating Graph Analytics by Co-Optimizing Storage and Access on an FPGA-HMC Platform

Optimizing BCPNN Learning Rule for Memory Access

A Remedy to Compute-in-Memory with Dynamic Random Access Memory: 1FeFET-1C Technology for Neuro-Symbolic AI

FPGA-based Hyrbid Memory Emulation System

Fpga Implementation Of Hierarchical Memory Architecture For Network Processors

eBrainII: a 3 kW Realtime Custom 3D DRAM Integrated ASIC Implementation of a Biologically Plausible Model of a Human Scale Cortex

Hardware Optimizations of Dense Binary Hyperdimensional Computing: Rematerialization of Hypervectors, Binarized Bundling, and Combinational Associative Memory

Real-time multicompartment Hodgkin-Huxley neuron emulation on SoC FPGA