Abstract:Binary Neural Network (BNN) exhibits substantial potential for low-cost and energy-efficient deployment, notably in the application of BNN-facilitated object detection on embedded devices. Despite its high compression ratio, the optimization of BNN encounters a significant challenge due to the discrete nature of binary weights and activations. Achieving good generalization performance poses a notable obstacle in this regard. Previous BNN training schemes that only focus on minimizing the empirical loss can easily suffer from overfitting problem, yielding poor generalization capacity when being exposed to unseen data. Sharpness-Aware Minimization (SAM), which simultaneously minimizes the loss value and sharpness of the loss landscape, has emerged as an effective approach for enhancing the generalization ability of neural networks, and has been demonstrated to be effective in improving the performance of BNNs designed for classification tasks. However, their work does not address the optimization challenge of applying SAM to models that handle multiple tasks, such as binary object detection, which involves both classification and location tasks. To address this issue, we propose a modified SAM scheme, denoted as BNN-SAM, which introduces a new objective allowing for the direct calculation of the optimal update vector. Moreover, the proposed scheme fosters multi-task optimization by establishing a common global flat minima for all the concerned tasks. This attribute renders it particularly fitting for scenarios such as object detection, inherently necessitating the joint optimization of both classification and localization. Comprehensive experiments on the PASCAL VOC and MS COCO datasets have shown that BNN-SAM can easily improve the performance of a baseline binary SSD300 detector to outperform state-of-the-art binary detectors, including BiDet, AutoBiDet and LWS-Det, by 6.5%, 5.0%, 1.1%, respectively, without the need of extra optimization approaches. Code is available at https://github.com/Anonymous2740/BNN-SAM.

LC-BiDet: Laterally Connected Binary Detector with Efficient Image Processing

BiDet: An Efficient Binarized Object Detector

Learning Efficient Binarized Object Detectors With Information Compression

Shallow Binary Features Enhance the Robustness of Deep Convolutional Neural Networks

TA-BiDet: Task-aligned binary object detector

BNN-SAM: Improving generalization of binary object detector by Seeking Flat Minima

BiPointNet: Binary Neural Network for Point Clouds

Pixel-wise binary classification network for salient object detection

Towards Accurate Binary Neural Networks via Modeling Contextual Dependencies.

Flexible Network Binarization with Layer-Wise Priority.

Eye Localization Based on Weight Binarization Cascade Convolution Neural Network

Learning Channel-Wise Interactions for Binary Convolutional Neural Networks.

DPNet: Dual-Path Network for Real-Time Object Detection With Lightweight Attention

Binarizing Sparse Convolutional Networks for Efficient Point Cloud Analysis

RBCN: Rectified Binary Convolutional Networks for Enhancing the Performance of 1-bit DCNNs

Network Binarization via Contrastive Learning

Tiny-BDN: an Efficient and Compact Barcode Detection Network

DIGCN: A Dynamic Interaction Graph Convolutional Network Based on Learnable Proposals for Object Detection

Locally Assembled Binary (Lab) Feature With Feature-Centric Cascade For Fast And Accurate Face Detection

Distribution-sensitive Information Retention for Accurate Binary Neural Network

Learning Effective Binary Visual Representations with Deep Networks