Abstract:There is a pressing need to build an architecture that could subsume these networks undera unified framework that achieves both higher performance and less overhead. To this end, two fundamental issues are yet to be addressed. The first one is how to implement the back propagation when neuronal activations are discrete. The second one is how to remove the full-precision hidden weights in the training phase to break the bottlenecks of memory/computation consumption. To address the first issue, we present a multistep neuronal activation discretization method and a derivative approximation technique that enable the implementing the back propagation algorithm on discrete DNNs. While for the second issue, we propose a discrete state transition (DST) methodology to constrain the weights in a discrete space without saving the hidden weights. In this way, we build a unified framework that subsumes the binary or ternary networks as its special cases.More particularly, we find that when both the weights and activations become ternary values, the DNNs can be reduced to gated XNOR networks (or sparse binary networks) since only the event of non-zero weight and non-zero activation enables the control gate to start the XNOR logic operations in the original binary networks. This promises the event-driven hardware design for efficient mobile intelligence. We achieve advanced performance compared with state-of-the-art algorithms. Furthermore,the computational sparsity and the number of states in the discrete space can be flexibly modified to make it suitable for various hardware platforms.

Multi-thresholded neuron and it's application in multi-valued logic

Realization of Neural Networks for Basic Operation on Multi-Valued Logic

Ten-Valued Multiplication Based on 2-5 Mixed-Valued Coding

A SYMMETRIC TERNARY LOGIC BASED MULTI-VALUED LEARNING NETWORK

Circuit Implementation of Multi-Thresholded Neuron (MTN) Using BiCMOS Technology

Multi-Valued Logic Learning Control

Design of D Flip-Flops Based on Multi-Threshold Neuron

Dynamics Analysis and Analog Associative Memory of Networks with LT Neurons.

Multi-Valued Neural Networks I A Multi-Valued Associative Memory

Multi-valued Logic Design Methodology with Double Negative Differential Resistance Transistors

Multi-valued Quantum Neurons

Possibility Of Combined Use Of Neuron-Mos And Rtd In Multi-Valued Logic Circuits

Simulations of threshold logic unit problems using memristor based synapses and CMOS neuron

Hybrid Dual-Complementary Metal-Oxide-semiconductor/memristor Synapse-Based Neural Network with Its Applications in Image Super-Resolution

Interpretable neural networks based on continuous-valued logic and multicriteria decision operators

Perceptron Implementation of Triple-Valued Logic Operations

Gated XNOR Networks: Deep Neural Networks with Ternary Weights and Activations under a Unified Discretization Framework.

Dual-mode Dendritic Devices Enhanced Neural Network Based on Electrolyte Gated Transistors

Logic Computing with Stateful Neural Networks of Resistive Switches

Reconfigurable Multiple-Valued Logic Function and Sequential Circuit Realizations via Threshold Logic Gates

A MoS2-based coplanar neuron transistor for logic applications