Abstract:With recent advancing of Internet of Things (IoTs), it becomes very attractive to implement the deep convolutional neural networks (DCNNs) onto embedded/portable systems. Presently, executing the software-based DCNNs requires high-performance server clusters in practice, restricting their widespread deployment on the mobile devices. To overcome this issue, considerable research efforts have been conducted in the context of developing highly-parallel and specific DCNN hardware, utilizing GPGPUs, FPGAs, and ASICs. Stochastic Computing (SC), which uses bit-stream to represent a number within [-1, 1] by counting the number of ones in the bit-stream, has a high potential for implementing DCNNs with high scalability and ultra-low hardware footprint. Since multiplications and additions can be calculated using AND gates and multiplexers in SC, significant reductions in power/energy and hardware footprint can be achieved compared to the conventional binary arithmetic implementations. The tremendous savings in power (energy) and hardware resources bring about immense design space for enhancing scalability and robustness for hardware DCNNs. This paper presents the first comprehensive design and optimization framework of SC-based DCNNs (SC-DCNNs). We first present the optimal designs of function blocks that perform the basic operations, i.e., inner product, pooling, and activation function. Then we propose the optimal design of four types of combinations of basic function blocks, named feature extraction blocks, which are in charge of extracting features from input feature maps. Besides, weight storage methods are investigated to reduce the area and power/energy consumption for storing weights. Finally, the whole SC-DCNN implementation is optimized, with feature extraction blocks carefully selected, to minimize area and power/energy consumption while maintaining a high network accuracy level.

Stochastic configuration networks with CPU-GPU implementation for large-scale data analytics

Hybrid Parallel Stochastic Configuration Networks for Industrial Data Analytics

Stochastic Configuration Networks Ensemble with Heterogeneous Features for Large-Scale Data Analytics

Distributed Stochastic Configuration Networks with Cooperative Learning Paradigm

Distributed Sparse Learning for Stochastic Configuration Networks Via Alternating Direction Method of Multipliers

Deep Stochastic Configuration Networks with Different Random Sampling Strategies

2-D Stochastic Configuration Networks for Image Data Analytics

Two Dimensional Stochastic Configuration Networks for Image Data Analytics

Deeper Insights into Learning Performance of Stochastic Configuration Networks

Stochastic Configuration Machines for Industrial Artificial Intelligence

Evolutionary Stochastic Configuration Networks for Industrial Data Analytics

Lightweight Stochastic Configuration Networks

Stochastic Configuration Networks: Fundamentals and Algorithms

Stochastic configuration networks with block increments for data modeling in process industries

Stochastic Configuration Machines: FPGA Implementation

SC-DCNN: Highly-Scalable Deep Convolutional Neural Network using Stochastic Computing

Orthogonal Stochastic Configuration Networks with Adaptive Construction Parameter for Data Analytics

Greedy stochastic configuration networks for ill-posed problems

A Lightweight Learning Method for Stochastic Configuration Networks Using Non-Inverse Solution

Enhancing Deep Stochastic Configuration Networks: Efficient Training via Low-Rank Matrix Approximation

Driving Amount Based Stochastic Configuration Network for Industrial Process Modeling.