Abstract:With recent advancing of Internet of Things (IoTs), it becomes very attractive to implement the deep convolutional neural networks (DCNNs) onto embedded/portable systems. Presently, executing the software-based DCNNs requires high-performance server clusters in practice, restricting their widespread deployment on the mobile devices. To overcome this issue, considerable research efforts have been conducted in the context of developing highly-parallel and specific DCNN hardware, utilizing GPGPUs, FPGAs, and ASICs. Stochastic Computing (SC), which uses bit-stream to represent a number within [-1, 1] by counting the number of ones in the bit-stream, has a high potential for implementing DCNNs with high scalability and ultra-low hardware footprint. Since multiplications and additions can be calculated using AND gates and multiplexers in SC, significant reductions in power/energy and hardware footprint can be achieved compared to the conventional binary arithmetic implementations. The tremendous savings in power (energy) and hardware resources bring about immense design space for enhancing scalability and robustness for hardware DCNNs. This paper presents the first comprehensive design and optimization framework of SC-based DCNNs (SC-DCNNs). We first present the optimal designs of function blocks that perform the basic operations, i.e., inner product, pooling, and activation function. Then we propose the optimal design of four types of combinations of basic function blocks, named feature extraction blocks, which are in charge of extracting features from input feature maps. Besides, weight storage methods are investigated to reduce the area and power/energy consumption for storing weights. Finally, the whole SC-DCNN implementation is optimized, with feature extraction blocks carefully selected, to minimize area and power/energy consumption while maintaining a high network accuracy level.

Stochastic Computing for Sub-0.2V Applications-Technology Evaluation and Design Optimization

Design Guidelines of Stochastic Computing Based on FinFET: A Technology-Circuit Perspective

Stochastic Computing Hardware Design and Optimization for Convolutional Neutral Networks

Noise-aware DVFS Transition Sequence Optimization for Battery-Powered IoT Devices.

Principles of Stochastic Computing: Fundamental Concepts and Applications

Computing-in-memory Circuits and Cross-Layer Integrated Design and Optimization: from SRAM to FeFET

How Close to the CMOS Voltage Scaling Limit for FinFET Technology? — Near-threshold Computing and Stochastic Computing

From Multipliers to Integrators: A Survey of Stochastic Computing Primitives

Circuit Reliability Comparison Between Stochastic Computing and Binary Computing.

A Parallel Bitstream Generator for Stochastic Computing

Investigation on Stochastic Computing Neuron Circuit Based on FinFET Technology

Stochastic Synthesis for Stochastic Computing

Systematical Investigation of Flicker Noise in 14 nm FinFET Devices towards Stochastic Computing Application

A 28-Nm 198.9-TOPS/W Fault-Tolerant Stochastic Computing Neural Network Processor

SC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic Computing

Stochastic Computing Convolution Neural Network Architecture Reinvented For Highly Efficient Artificial Intelligence Workload on Field Programmable Gate Array

Investigation on the Vdd Scaling Limit of Stochastic Computing Circuits Based on FinFET Technology

Not Your Father’s Stochastic Computing (SC)! Efficient Yet Accurate End-to-End SC Accelerator Design

Steep Slope Devices: Enabling New Architectural Paradigms

Noise-Aware DVFS for Efficient Transitions on Battery-Powered IoT Devices

Simultaneous Area and Latency Optimization for Stochastic Circuits by D Flip-Flop Insertion