Abstract:Artificial neural networks (ANNs), like convolutional neural networks (CNNs), have achieved the state-of-the-art results for many machine learning tasks. However, inference with large-scale full-precision CNNs must cause substantial energy consumption and memory occupation, which seriously hinders their deployment on mobile and embedded systems. Highly inspired from biological brain, spiking neural networks (SNNs) are emerging as new solutions because of natural superiority in brain-like learning and great energy efficiency with event-driven communication and computation. Nevertheless, training a deep SNN remains a main challenge and there is usually a big accuracy gap between ANNs and SNNs. In this paper, we introduce a hardware-friendly conversion algorithm called “scatter-and-gather” to convert quantized ANNs to lossless SNNs, where neurons are connected with ternary {−1,0,1} synaptic weights. Each spiking neuron is stateless and more like original McCulloch and Pitts model, because it fires at most one spike and need be reset at each time step. Furthermore, we develop an incremental mapping framework to demonstrate efficient network deployments on a reconfigurable neuromorphic chip. Experimental results show our spiking LeNet on MNIST and VGG-Net on CIFAR-10 datasetobtain 99.37% and 91.91% classification accuracy, respectively. Besides, the presented mapping algorithm manages network deployment on our neuromorphic chip with maximum resource efficiency and excellent flexibility. Our four-spike LeNet and VGG-Net on chip can achieve respective real-time inference speed of 0.38 ms/image, 3.24 ms/image, and an average power consumption of 0.28 mJ/image and 2.3 mJ/image at 0.9 V, 252 MHz, which is nearly two orders of magnitude more efficient than traditional GPUs.

A Neural Network Hardware Model for Image Thinning

Neural network model processing method and device, equipment and storage medium

Binary fingerprint image thinning using template-based PCNNs.

Loss Constrains Added Squeeze and Excitation Blocks for Pruning Deep Neural Networks

Efficient Structure Slimming for Spiking Neural Networks

Learning Efficient Convolutional Networks Through Network Slimming.

Towards Thinner Convolutional Neural Networks Through Gradually Global Pruning

TB-DNN: A Thin Binarized Deep Neural Network with High Accuracy

Efficient DNN Neuron Pruning by Minimizing Layer-wise Nonlinear Reconstruction Error

Deep Adaptive Network: An Efficient Deep Neural Network with Sparse Binary Connections

ThiNet: Pruning CNN Filters for a Thinner Net

Small Object Remover Cnn And Robustness Designs

A novel weight pruning strategy for light weight neural networks with application to the diagnosis of skin disease

Hybrid Parallel Image Processing Algorithm for Binary Images with Image Thinning Technique

Prune the Convolutional Neural Networks with Sparse Shrink

Towards Efficient Neural Networks On-a-chip: Joint Hardware-Algorithm Approaches

A Scatter-and-Gather Spiking Convolutional Neural Network on a Reconfigurable Neuromorphic Hardware

Identifying and Pruning Redundant Structures for Deep Neural Networks

A Compact Parallel Pruning Scheme for Deep Learning Model and Its Mobile Instrument Deployment

New Pruning Method Based on DenseNet Network for Image Classification

DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks