Abstract:Binary neural network (BNN), where both the weight and the activation values are represented with one bit, provides an attractive alternative to deploy highly efficient deep learning inference on resource-constrained edge devices. However, our investigation reveals that, to achieve satisfactory accuracy gains, state-of-the-art (SOTA) BNNs, such as FracBNN and ReActNet, usually have to incorporate various auxiliary floating-point components and increase the model size, which in turn degrades the hardware performance efficiency. In this paper, we aim to quantify such hardware inefficiency in SOTA BNNs and further mitigate it with negligible accuracy loss. First, we observe that the auxiliary floating-point (AFP) components consume an average of 93% DSPs, 46% LUTs, and 62% FFs, among the entire BNN accelerator resource utilization. To mitigate such overhead, we propose a novel algorithm-hardware co-design, called FuseBNN , to fuse those AFP operators without hurting the accuracy. On average, FuseBNN reduces AFP resource utilization to 59% DSPs, 13% LUTs, and 16% FFs. Second, SOTA BNNs often use the compact MobileNetV1 as the backbone network but have to replace the lightweight 3 × 3 depth-wise convolution (DWC) with the 3 × 3 standard convolution (SC, e.g., in ReActNet and our ReActNet-adapted BaseBNN) or even more complex fractional 3 × 3 SC (e.g., in FracBNN) to bridge the accuracy gap. As a result, the model parameter size is significantly increased and becomes 2.25 × larger than that of the 4-bit direct quantization with the original DWC (4-Bit-Net); the number of multiply-accumulate operations is also significantly increased so that the overall LUT resource usage of BaseBNN is almost the same as that of 4-Bit-Net. To address this issue, we propose HyBNN , where we binarize depth-wise separation convolution (DSC) blocks for the first time to decrease the model size and incorporate 4-bit DSC blocks to compensate for the accuracy loss. For the ship detection task in synthetic aperture radar imagery on the AMD-Xilinx ZCU102 FPGA, HyBNN achieves a detection accuracy of 94.8% and a detection speed of 615 frames per second (FPS), which is 6.8 × faster than FuseBNN+ (94.9% accuracy) and 2.7 × faster than 4-Bit-Net (95.9% accuracy). For image classification on the CIFAR-10 dataset on the AMD-Xilinx Ultra96-V2 FPGA, HyBNN achieves 1.5 × speedup and 0.7% better accuracy over SOTA FracBNN.

BinSparX: Sparsified Binary Neural Networks for Reduced Hardware Non-Idealities in Xbar Arrays

Sparsity-Aware Optimization of In-Memory Bayesian Binary Neural Network Accelerators

An Efficient Channel-Aware Sparse Binarized Neural Networks Inference Accelerator

Crossbar-Aligned & Integer-Only Neural Network Compression for Efficient In-Memory Acceleration

SpikeSim: An end-to-end Compute-in-Memory Hardware Evaluation Tool for Benchmarking Spiking Neural Networks

Journal Track Paper ICFPT 2023 : HyBNN: Quantifying and Optimizing Hardware Efficiency of Binary Neural Networks

SoBS-X: Squeeze-Out Bit Sparsity for ReRAM-Crossbar-Based Neural Network Accelerator.

Signed Binarization: Unlocking Efficiency Through Repetition-Sparsity Trade-Off

CBin-NN: An Inference Engine for Binarized Neural Networks

HyBNN: Quantifying and Optimizing Hardware Efficiency of Binary Neural Networks.

Batch Normalization-Free Weight-Binarized SNN Based on Hardware-Saving IF Neuron.

Parallelizing SRAM Arrays with Customized Bit-Cell for Binary Neural Networks

BASER: Bit-wise Approximate Compressor Configurable In-SRAM-computing for Energy-Efficient Neural Network Acceleration with Data-aware Weight Remapping Method

Exploiting Kernel Compression on BNNs

An Approach of Binary Neural Network Energy-Efficient Implementation

A High-Efficiency FPGA-Based Accelerator for Binarized Neural Network

TCP-Net: Minimizing Operation Counts of Binarized Neural Network Inference.

Sub-bit Neural Networks: Learning to Compress and Accelerate Binary Neural Networks

Sparsity-Inducing Binarized Neural Networks

SME: ReRAM-based Sparse-Multiplication-Engine to Squeeze-Out Bit Sparsity of Neural Network

Parapim: A Parallel Processing-In-Memory Accelerator For Binary-Weight Deep Neural Networks