Abstract:Binary neural network (BNN), where both the weight and the activation values are represented with one bit, provides an attractive alternative to deploy highly efficient deep learning inference on resource-constrained edge devices. However, our investigation reveals that, to achieve satisfactory accuracy gains, state-of-the-art (SOTA) BNNs, such as FracBNN and ReActNet, usually have to incorporate various auxiliary floating-point components and increase the model size, which in turn degrades the hardware performance efficiency. In this paper, we aim to quantify such hardware inefficiency in SOTA BNNs and further mitigate it with negligible accuracy loss. First, we observe that the auxiliary floating-point (AFP) components consume an average of 93% DSPs, 46% LUTs, and 62% FFs, among the entire BNN accelerator resource utilization. To mitigate such overhead, we propose a novel algorithm-hardware co-design, called FuseBNN , to fuse those AFP operators without hurting the accuracy. On average, FuseBNN reduces AFP resource utilization to 59% DSPs, 13% LUTs, and 16% FFs. Second, SOTA BNNs often use the compact MobileNetV1 as the backbone network but have to replace the lightweight 3 × 3 depth-wise convolution (DWC) with the 3 × 3 standard convolution (SC, e.g., in ReActNet and our ReActNet-adapted BaseBNN) or even more complex fractional 3 × 3 SC (e.g., in FracBNN) to bridge the accuracy gap. As a result, the model parameter size is significantly increased and becomes 2.25 × larger than that of the 4-bit direct quantization with the original DWC (4-Bit-Net); the number of multiply-accumulate operations is also significantly increased so that the overall LUT resource usage of BaseBNN is almost the same as that of 4-Bit-Net. To address this issue, we propose HyBNN , where we binarize depth-wise separation convolution (DSC) blocks for the first time to decrease the model size and incorporate 4-bit DSC blocks to compensate for the accuracy loss. For the ship detection task in synthetic aperture radar imagery on the AMD-Xilinx ZCU102 FPGA, HyBNN achieves a detection accuracy of 94.8% and a detection speed of 615 frames per second (FPS), which is 6.8 × faster than FuseBNN+ (94.9% accuracy) and 2.7 × faster than 4-Bit-Net (95.9% accuracy). For image classification on the CIFAR-10 dataset on the AMD-Xilinx Ultra96-V2 FPGA, HyBNN achieves 1.5 × speedup and 0.7% better accuracy over SOTA FracBNN.

Fully Binarized Convolutional Neural Network for Accelerating Edge Vision Computing

BitFlow-Net: Toward Fully Binarized Convolutional Neural Networks

Accelerating Binarized Convolutional Neural Networks with Software-Programmable FPGAs.

A Fully Onchip Binarized Convolutional Neural Network FPGA Impelmentation with Accurate Inference.

BitFlow: Exploiting Vector Parallelism for Binary Neural Networks on CPU

A High-Efficiency FPGA-Based Accelerator for Binarized Neural Network

FBNA: A Fully Binarized Neural Network Accelerator.

Automated flow for compressing convolution neural networks for efficient edge-computation with FPGA

Accelerating Low Bit-Width Convolutional Neural Networks with Embedded FPGA.

BiPointNet: Binary Neural Network for Point Clouds

Binary Complex Neural Network Acceleration on FPGA

TB-DNN: A Thin Binarized Deep Neural Network with High Accuracy

An Approach of Binary Neural Network Energy-Efficient Implementation

Gradient Matters: Designing Binarized Neural Networks Via Enhanced Information-Flow

Energy-Efficient Machine Learning Accelerator for Binary Neural Networks

Journal Track Paper ICFPT 2023 : HyBNN: Quantifying and Optimizing Hardware Efficiency of Binary Neural Networks

Binary Neural Networks as a general-propose compute paradigm for on-device computer vision

A high-throughput scalable BNN accelerator with fully pipelined architecture

TCP-Net: Minimizing Operation Counts of Binarized Neural Network Inference.

An Efficient Channel-Aware Sparse Binarized Neural Networks Inference Accelerator

HyBNN: Quantifying and Optimizing Hardware Efficiency of Binary Neural Networks.