Abstract:Due to less memory requirement, low computation overhead and negligible accuracy degradation, deep neural networks with binary/ternary weights (BTNNs) have been widely employed on low-power mobile and Internet of Things (IoT) devices with limited storage capacity. Some hardware implementations have been proposed to accelerate the inference of BTNNs by utilizing the multiplication-free feature. However, some implicit characteristics in BTNN convolution, such as high arithmetic complexity and numerous redundant operations, are never considered. In this paper, we propose four optimization techniques to fully exploit these features. First, a feature-integral-based convolution (FIBC) method is proposed to reduce the arithmetic complexity of convolutional layers. Second, a kernel-transformation-feature-reconstruction (KTFR) convolution method is presented to remove redundant operations in BTNN convolution. Third, a hierarchical load-balancing mechanism (HLBM) is designed to eliminate zero value computation and improve resource utilization. Finally, a joint optimization approach for convolutional layers is proposed to search optimal calculation pattern for each layer. Based on the proposed four techniques, we design a reconfigurable processor in a 28-nm CMOS technology to accelerate the inferences of BTNNs. The four proposed techniques improve energy efficiency by 2.07 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> , 1.65 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> , 1.25 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> , and 2.24 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times $ </tex-math></inline-formula> for BTNNs respectively, compared with the baseline implementation which disables the proposed techniques. Benchmarked with binary-weight AlexNet, the processor achieves an energy efficiency of 19.9 TOPS/W at 200 MHz and 0.9 V.

Bisection Neural Network Toward Reconfigurable Hardware Implementation

Reconfigurable Binary Neural Network Accelerator with Adaptive Parallelism Scheme

Cerebron: A Reconfigurable Architecture for Spatiotemporal Sparse Spiking Neural Networks

An Energy-Efficient Reconfigurable Processor for Binary-and Ternary-Weight Neural Networks With Flexible Data Bit Width

An Approach of Binary Neural Network Energy-Efficient Implementation

Neural Network-Hardware Co-design for Scalable RRAM-based BNN Accelerators

A&B BNN: Add&Bit-Operation-Only Hardware-Friendly Binary Neural Network

STBNN: Hardware-friendly Spatio-Temporal Binary Neural Network with High Pattern Recognition Accuracy

ReBNet: Residual Binarized Neural Network

O3BNN-R: An Out-of-Order Architecture for High-Performance and Regularized BNN Inference

A 617-TOPS/W All-Digital Binary Neural Network Accelerator in 10-nm FinFET CMOS

Batch Normalization-Free Weight-Binarized SNN Based on Hardware-Saving IF Neuron.

Hardware Platform-Aware Binarized Neural Network Model Optimization

Resilient Binary Neural Network.

An Improved RRAM-Based Binarized Neural Network with High Variation-Tolerated Forward/Backward Propagation Module

A New Hardware Implementation Approach Of Bnns Based On Nonlinear 2t2r Synaptic Cell

FCA-BNN: Flexible and Configurable Accelerator for Binarized Neural Networks on FPGA.

Binary Neural Networks in FPGAs: Architectures, Tool Flows and Hardware Comparisons

A Scatter-and-Gather Spiking Convolutional Neural Network on a Reconfigurable Neuromorphic Hardware

FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations.

Exploiting Kernel Compression on BNNs