Abstract:Binary neural network (BNN), where both the weight and the activation values are represented with one bit, provides an attractive alternative to deploy highly efficient deep learning inference on resource-constrained edge devices. However, our investigation reveals that, to achieve satisfactory accuracy gains, state-of-the-art (SOTA) BNNs, such as FracBNN and ReActNet, usually have to incorporate various auxiliary floating-point components and increase the model size, which in turn degrades the hardware performance efficiency. In this paper, we aim to quantify such hardware inefficiency in SOTA BNNs and further mitigate it with negligible accuracy loss. First, we observe that the auxiliary floating-point (AFP) components consume an average of 93% DSPs, 46% LUTs, and 62% FFs, among the entire BNN accelerator resource utilization. To mitigate such overhead, we propose a novel algorithm-hardware co-design, called FuseBNN , to fuse those AFP operators without hurting the accuracy. On average, FuseBNN reduces AFP resource utilization to 59% DSPs, 13% LUTs, and 16% FFs. Second, SOTA BNNs often use the compact MobileNetV1 as the backbone network but have to replace the lightweight 3 × 3 depth-wise convolution (DWC) with the 3 × 3 standard convolution (SC, e.g., in ReActNet and our ReActNet-adapted BaseBNN) or even more complex fractional 3 × 3 SC (e.g., in FracBNN) to bridge the accuracy gap. As a result, the model parameter size is significantly increased and becomes 2.25 × larger than that of the 4-bit direct quantization with the original DWC (4-Bit-Net); the number of multiply-accumulate operations is also significantly increased so that the overall LUT resource usage of BaseBNN is almost the same as that of 4-Bit-Net. To address this issue, we propose HyBNN , where we binarize depth-wise separation convolution (DSC) blocks for the first time to decrease the model size and incorporate 4-bit DSC blocks to compensate for the accuracy loss. For the ship detection task in synthetic aperture radar imagery on the AMD-Xilinx ZCU102 FPGA, HyBNN achieves a detection accuracy of 94.8% and a detection speed of 615 frames per second (FPS), which is 6.8 × faster than FuseBNN+ (94.9% accuracy) and 2.7 × faster than 4-Bit-Net (95.9% accuracy). For image classification on the CIFAR-10 dataset on the AMD-Xilinx Ultra96-V2 FPGA, HyBNN achieves 1.5 × speedup and 0.7% better accuracy over SOTA FracBNN.

FBNA: A Fully Binarized Neural Network Accelerator.

A High-Efficiency FPGA-Based Accelerator for Binarized Neural Network

FCA-BNN: Flexible and Configurable Accelerator for Binarized Neural Networks on FPGA.

A Convolutional Neural Network Accelerator Architecture with Fine-Granular Mixed Precision Configurability.

An Efficient Channel-Aware Sparse Binarized Neural Networks Inference Accelerator

An Energy-Efficient Bagged Binary Neural Network Accelerator

Energy-Efficient Machine Learning Accelerator for Binary Neural Networks

Accelerating Binarized Convolutional Neural Networks with Software-Programmable FPGAs.

FP-BNN: Binarized neural network on FPGA

A high-throughput scalable BNN accelerator with fully pipelined architecture

CBin-NN: An Inference Engine for Binarized Neural Networks

HyBNN: Quantifying and Optimizing Hardware Efficiency of Binary Neural Networks.

FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations.

An Approach of Binary Neural Network Energy-Efficient Implementation

Journal Track Paper ICFPT 2023 : HyBNN: Quantifying and Optimizing Hardware Efficiency of Binary Neural Networks

BitFlow-Net: Toward Fully Binarized Convolutional Neural Networks

Network Binarization via Contrastive Learning

Reconfigurable Binary Neural Network Accelerator with Adaptive Parallelism Scheme

A Fully Onchip Binarized Convolutional Neural Network FPGA Impelmentation with Accurate Inference.

Fully Binarized Convolutional Neural Network for Accelerating Edge Vision Computing

FTBNN: Rethinking Non-linearity for 1-bit CNNs and Going Beyond