Abstract:Deep convolutional neural networks (DCNNs) are one of the most promising deep learning techniques and have been recognized as the dominant approach for almost all recognition and detection tasks. The computation of DCNNs is memory intensive due to large feature maps and neuron connections, and the performance highly depends on the capability of hardware resources. With the recent trend of wearable devices and Internet of Things, it becomes desirable to integrate the DCNNs onto embedded and portable devices that require low power and energy consumptions and small hardware footprints. Recently stochastic computing (SC)-DCNN demonstrated that SC as a low-cost substitute to binary-based computing radically simplifies the hardware implementation of arithmetic units and has the potential to satisfy the stringent power requirements in embedded devices. In SC, many arithmetic operations that are resource-consuming in binary designs can be implemented with very simple hardware logic, alleviating the extensive computational complexity. It offers a colossal design space for integration and optimization due to its reduced area and soft error resiliency. In this paper, we present HEIF, a highly efficient SC-based inference framework of the large-scale DCNNs, with broad applications including (but not limited to) <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">LeNet-5 and AlexNet</italic> , that achieves high energy efficiency and low area/hardware cost. Compared to SC-DCNN, HEIF features: 1) the first (to the best of our knowledge) SC-based rectified linear unit activation function to catch up with the recent advances in software models and mitigate degradation in application-level accuracy; 2) the redesigned approximate parallel counter and optimized stochastic multiplication using transmission gates and inverse mirror adders; and 3) the new optimization of weight storage using clustering. Most importantly, to achieve maximum energy efficiency while maintaining acceptable accuracy, HEIF considers holistic optimizations on cascade connection of function blocks in DCNN, pipelining technique, and bit-stream length reduction. Experimental results show that in large-scale applications HEIF outperforms previous SC-DCNN by the throughput of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4.1\times $ </tex-math></inline-formula> , by area efficiency of up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$6.5\times $ </tex-math></inline-formula> , and achieves up to <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">${5.6\times }$ </tex-math></inline-formula> energy improvement.

Towards Budget-Driven Hardware Optimization for Deep Convolutional Neural Networks Using Stochastic Computing

DSCNN: Hardware-oriented Optimization for Stochastic Computing Based Deep Convolutional Neural Networks.

Structural design optimization for deep convolutional neural networks using stochastic computing.

Towards Acceleration of Deep Convolutional Neural Networks Using Stochastic Computing

SC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic Computing

Fully-Parallel Area-Efficient Deep Neural Network Design Using Stochastic Computing

Stochastic Computing Hardware Design and Optimization for Convolutional Neutral Networks

Hardware-driven Nonlinear Activation for Stochastic Computing Based Deep Convolutional Neural Networks.

A Low-Cost Implementation Method on Deep Neural Network Using Stochastic Computing

Optimization for Deep Convolutional Neural Network of Stochastic Computing on MLC-PCM-based System

Optimizing Stochastic Computing for Low Latency Inference of Convolutional Neural Networks

Designing Reconfigurable Large-Scale Deep Learning Systems Using Stochastic Computing

Stochastic Computing Convolution Neural Network Architecture Reinvented For Highly Efficient Artificial Intelligence Workload on Field Programmable Gate Array

An area and energy efficient design of domain-wall memory-based deep convolutional neural networks using stochastic computing

HEIF: Highly Efficient Stochastic Computing based Inference Framework for Deep Neural Networks

Softmax Regression Design for Stochastic Computing Based Deep Convolutional Neural Networks.

High Performance and Hardware Efficient Stochastic Computing Elements for Deep Neural Network

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

Hybrid Stochastic-Binary Computing for Low-Latency and High-Precision Inference of CNNs

Efficient Fast Convolution Architecture Based on Stochastic Computing

Reconfigurable Spatial-Parallel Stochastic Computing for Accelerating Sparse Convolutional Neural Networks