Abstract:With recent advancing of Internet of Things (IoTs), it becomes very attractive to implement the deep convolutional neural networks (DCNNs) onto embedded/portable systems. Presently, executing the software-based DCNNs requires high-performance server clusters in practice, restricting their widespread deployment on the mobile devices. To overcome this issue, considerable research efforts have been conducted in the context of developing highly-parallel and specific DCNN hardware, utilizing GPGPUs, FPGAs, and ASICs. Stochastic Computing (SC), which uses bit-stream to represent a number within [-1, 1] by counting the number of ones in the bit-stream, has a high potential for implementing DCNNs with high scalability and ultra-low hardware footprint. Since multiplications and additions can be calculated using AND gates and multiplexers in SC, significant reductions in power/energy and hardware footprint can be achieved compared to the conventional binary arithmetic implementations. The tremendous savings in power (energy) and hardware resources bring about immense design space for enhancing scalability and robustness for hardware DCNNs. This paper presents the first comprehensive design and optimization framework of SC-based DCNNs (SC-DCNNs). We first present the optimal designs of function blocks that perform the basic operations, i.e., inner product, pooling, and activation function. Then we propose the optimal design of four types of combinations of basic function blocks, named feature extraction blocks, which are in charge of extracting features from input feature maps. Besides, weight storage methods are investigated to reduce the area and power/energy consumption for storing weights. Finally, the whole SC-DCNN implementation is optimized, with feature extraction blocks carefully selected, to minimize area and power/energy consumption while maintaining a high network accuracy level.

Parallel Hybrid Stochastic-Binary-Based Neural Network Accelerators

Parallel Convolutional Neural Network (CNN) Accelerators Based on Stochastic Computing

Stochastic-Binary Hybrid Spatial Coding Multiplier for Convolutional Neural Network Accelerator

When Sorting Network Meets Parallel Bitstreams: A Fault-Tolerant Parallel Ternary Neural Network Accelerator Based on Stochastic Computing.

Accurate and Energy-Efficient Implementation of Non-Linear Adder in Parallel Stochastic Computing Using Sorting Network

Hybrid Stochastic-Binary Computing for Low-Latency and High-Precision Inference of CNNs

A Parallel Bitstream Generator for Stochastic Computing

Optimizing Stochastic Computing for Low Latency Inference of Convolutional Neural Networks

An Energy-Efficient Mixed-Signal Parallel Multiply-Accumulate (MAC) Engine Based on Stochastic Computing

Efficient Parallel Stochastic Computing Multiply-Accumulate (MAC) Technique Using Pseudo-Sobol Bit-Streams

A 28-Nm 198.9-TOPS/W Fault-Tolerant Stochastic Computing Neural Network Processor

Accurate yet Efficient Stochastic Computing Neural Acceleration with High Precision Residual Fusion

Not Your Father’s Stochastic Computing (SC)! Efficient Yet Accurate End-to-End SC Accelerator Design

Neural Synaptic Plasticity-Inspired Computing: A High Computing Efficient Deep Convolutional Neural Network Accelerator

A Hybrid Heterogeneous Neural Network Accelerator Based on Systolic Array

Efficient Non-Linear Adder for Stochastic Computing with Approximate Spatial-Temporal Sorting Network

Reconfigurable Spatial-Parallel Stochastic Computing for Accelerating Sparse Convolutional Neural Networks

Efficient yet Accurate End-to-End SC Accelerator Design

An Energy-Efficient Bayesian Neural Network Implementation Using Stochastic Computing Method

SC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic Computing

Reconfigurable Binary Neural Network Accelerator with Adaptive Parallelism Scheme