Abstract:Simulation of large scale biologically plausible spiking neural networks, e.g., Bayesian Confidence Propagation Neural Network (BCPNN), usually requires high-performance supercomputers with dedicated accelerators, such as GPUs, FPGAs, or even Application-Specific Integrated Circuits (ASICs). Almost all of these computers are based on the von Neumann architecture that separates storage and computation. In all these solutions, memory access is the dominant cost even for highly customized computation and memory architecture, such as ASICs. In this paper, we propose an optimization technique that can make the BCPNN simulation memory access friendly by avoiding a dual-access pattern. The BCPNN synaptic traces and weights are organized as matrices accessed both row-wise and column-wise. Accessing data stored in DRAM with a dual-access pattern is extremely expensive. A post-synaptic history buffer and an approximation function thus are introduced to eliminate the troublesome column update. The error analysis combining theoretical analysis and experiments suggests that the probability of introducing intolerable errors by such optimization can be bounded to a very small number, which makes it almost negligible. Derivation and validation of such a bound is the core contribution of this paper. Experiments on a GPU platform shows that compared to the previously reported baseline simulation strategy, the proposed optimization technique reduces the storage requirement by 33%, the global memory access demand by more than 27% and DRAM access rate by more than 5%; the latency of updating synaptic traces decreases by roughly 50%. Compared with the other similar optimization technique reported in the literature, our method clearly shows considerably better results. Although the BCPNN is used as the targeted neural network model, the proposed optimization method can be applied to other artificial neural network models based on a Hebbian learning rule.

Exploiting Near-Memory Processing Architectures for Bayesian Neural Networks Acceleration

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

Sparsity-Aware Optimization of In-Memory Bayesian Binary Neural Network Accelerators

Shift-BNN: Highly-Efficient Probabilistic Bayesian Neural Network Training via Memory-Friendly Pattern Retrieving

Optimizing BCPNN Learning Rule for Memory Access

An Energy-Efficient Near-Data Processing Accelerator for DNNs that Optimizes Data Accesses

Bayes2IMC: In-Memory Computing for Bayesian Binary Neural Networks

An Energy-Efficient Architecture for Accelerating Inference of Memory-Augmented Neural Networks

An Efficient Channel-Aware Sparse Binarized Neural Networks Inference Accelerator

High-Performance FPGA-based Accelerator for Bayesian Neural Networks

A FPGA-based Hardware Accelerator for Bayesian Confidence Propagation Neural Network

Parapim: A Parallel Processing-In-Memory Accelerator For Binary-Weight Deep Neural Networks

Bayesian Inference Accelerator for Spiking Neural Networks

A Hybrid Precision Low Power Computing-in-memory Architecture for Neural Networks

Memory Access Optimization of a Neural Network Accelerator Based on Memory Controller

Accelerating Neural Network Inference with Processing-in-DRAM: From the Edge to the Cloud

Model Architecture Adaption for Bayesian Neural Networks

Accelerating Deep Neural Networks in Processing-in-Memory Platforms: Analog or Digital Approach?

An Improved Hardware Accelaration Architecture of Binary Neural Network With 1T1R Array Based Forward/Backward Propagation Module

NDPGNN: A Near-Data Processing Architecture for GNN Training and Inference Acceleration

A Survey of Near-Data Processing Architectures for Neural Networks