Abstract:In recent years, the scaling down that Moore’s Law relies on has been gradually slowing down, and the traditional von Neumann architecture has been limiting the improvement of computing power. Thus, neuromorphic in-memory computing hardware has been proposed and is becoming a promising alternative. However, there is still a long way to make it possible, and one of the problems is to provide an efficient, reliable, and achievable neural network for hardware implementation. In this paper, we proposed a two-layer fully connected spiking neural network based on binary MRAM (Magneto-resistive Random Access Memory) synapses with low hardware cost. First, the network used an array of multiple binary MRAM cells to store multi-bit fixed-point weight values. This helps to simplify the read/write circuit. Second, we used different kinds of spike encoders that ensure the sparsity of input spikes, to reduce the complexity of peripheral circuits, such as sense amplifiers. Third, we designed a single-step learning rule, which fit well with the fixed-point binary weights. Fourth, we replaced the traditional exponential Leak-Integrate-Fire (LIF) neuron model to avoid the massive cost of exponential circuits. The simulation results showed that, compared to other similar works, our SNN with 1184 neurons and 313,600 synapses achieved an accuracy of up to 90.6% in the MNIST recognition task with full-resolution (28 × 28) and full-bit-depth (8-bit) images. In the case of low-resolution (16 × 16) and black-white (1-bit) images, the smaller version of our network with 384 neurons and 32,768 synapses still maintained an accuracy of about 77%, extending its application to ultra-low-cost situations. Both versions need less than 30,000 samples to reach convergence, which is a >50% reduction compared to other similar networks. As for robustness, it is immune to the fluctuation of MRAM cell resistance.

Algorithm and hardware codesign of sparse binary network on-chip

DANoC: An Efficient Algorithm and Hardware Codesign of Deep Neural Networks on Chip.

Deep Adaptive Network: An Efficient Deep Neural Network with Sparse Binary Connections

DaDianNao: A Machine-Learning Supercomputer

A Computing Efficient Hardware Architecture for Sparse Deep Neural Network Computing

Towards Efficient Neural Networks On-a-chip: Joint Hardware-Algorithm Approaches

A Scatter-and-Gather Spiking Convolutional Neural Network on a Reconfigurable Neuromorphic Hardware

Exploring the Sparsity-Quantization Interplay on a Novel Hybrid SNN Event-Driven Architecture

Efficient DNN Algorithm Design and Hardware Acceleration for Low-Level Vision

Deep Spiking Binary Neural Network for Digital Neuromorphic Hardware

Minimizing Area and Energy of Deep Learning Hardware Design Using Collective Low Precision and Structured Compression

Enabling High Performance Deep Learning Networks on Embedded Systems

Extremely Sparse Networks Via Binary Augmented Pruning for Fast Image Classification.

Efficient Hardware Optimization Strategies For Deep Neural Networks Acceleration Chip

Highly Efficient Sparse Neural Network Computing - Hardware and Software Solutions.

A Cost-Efficient High-Speed VLSI Architecture for Spiking Convolutional Neural Network Inference Using Time-Step Binary Spike Maps

Cambricon-S: Addressing Irregularity in Sparse Neural Networks Through A Cooperative Software/Hardware Approach.

Accelerated Inference Framework of Sparse Neural Network Based on Nested Bitmask Structure.

Special Topic on Nonvolatile Memory for Efficient Implementation of Neural/Neuromorphic Computing

A Low-Cost Hardware-Friendly Spiking Neural Network Based on Binary MRAM Synapses, Accelerated Using In-Memory Computing

Sparsely-Connected Neural Networks: Towards Efficient VLSI Implementation of Deep Neural Networks