Abstract:Due to less memory requirement, low computation overhead and negligible accuracy degradation, deep neural networks with binary/ternary weights (BTNNs) have been widely employed on low-power mobile and Internet of Things (IoT) devices with limited storage capacity. Some hardware implementations have been proposed to accelerate the inference of BTNNs by utilizing the multiplication-free feature. However, some implicit characteristics in BTNN convolution, such as high arithmetic complexity and numerous redundant operations, are never considered. In this paper, we propose four optimization techniques to fully exploit these features. First, a feature-integral-based convolution (FIBC) method is proposed to reduce the arithmetic complexity of convolutional layers. Second, a kernel-transformation-feature-reconstruction (KTFR) convolution method is presented to remove redundant operations in BTNN convolution. Third, a hierarchical load-balancing mechanism (HLBM) is designed to eliminate zero value computation and improve resource utilization. Finally, a joint optimization approach for convolutional layers is proposed to search optimal calculation pattern for each layer. Based on the proposed four techniques, we design a reconfigurable processor in a 28-nm CMOS technology to accelerate the inferences of BTNNs. The four proposed techniques improve energy efficiency by 2.07 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> , 1.65 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> , 1.25 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> , and 2.24 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> for BTNNs respectively, compared with the baseline implementation which disables the proposed techniques. Benchmarked with binary-weight AlexNet, the processor achieves an energy efficiency of 19.9 TOPS/W at 200 MHz and 0.9 V.

Tbn: Convolutional Neural Network With Ternary Inputs And Binary Weights

An Efficient Kernel Transformation Architecture for Binary- and Ternary-Weight Neural Network Inference.

FATNN: Fast and Accurate Ternary Neural Networks

Ternary Weight Networks

An In-Memory-Computing Charge-Domain Ternary CNN Classifier

SATB-Nets: Training Deep Neural Networks with Segmented Asymmetric Ternary and Binary Weights

Asymmetric Ternary Networks

TB-DNN: A Thin Binarized Deep Neural Network with High Accuracy

Simultaneously Optimizing Weight and Quantizer of Ternary Neural Network using Truncated Gaussian Approximation

Trained Ternary Quantization

Three-Means Ternary Quantization

Learning Sparse & Ternary Neural Networks with Entropy-Constrained Trained Ternarization (EC2T)

Binary Convolutional Neural Network with High Accuracy and Compression Rate

Build a Compact Binary Neural Network Through Bit-level Sensitivity and Data Pruning.

TernaryNet: faster deep model inference without GPUs for medical 3D segmentation using sparse and binary convolutions

An Energy-Efficient Reconfigurable Processor for Binary-and Ternary-Weight Neural Networks With Flexible Data Bit Width

Training Compact Neural Networks with Binary Weights and Low Precision Activations

Binary Volumetric Convolutional Neural Networks for 3-D Object Recognition

Separable Binary Convolutional Neural Network on Embedded Systems

A Time-Domain Binary CNN Engine with Error-Detection-Based Resilience in 28nm CMOS

Basic Binary Convolution Unit for Binarized Image Restoration Network