Abstract:Binary Neural Network (BNN) exhibits substantial potential for low-cost and energy-efficient deployment, notably in the application of BNN-facilitated object detection on embedded devices. Despite its high compression ratio, the optimization of BNN encounters a significant challenge due to the discrete nature of binary weights and activations. Achieving good generalization performance poses a notable obstacle in this regard. Previous BNN training schemes that only focus on minimizing the empirical loss can easily suffer from overfitting problem, yielding poor generalization capacity when being exposed to unseen data. Sharpness-Aware Minimization (SAM), which simultaneously minimizes the loss value and sharpness of the loss landscape, has emerged as an effective approach for enhancing the generalization ability of neural networks, and has been demonstrated to be effective in improving the performance of BNNs designed for classification tasks. However, their work does not address the optimization challenge of applying SAM to models that handle multiple tasks, such as binary object detection, which involves both classification and location tasks. To address this issue, we propose a modified SAM scheme, denoted as BNN-SAM, which introduces a new objective allowing for the direct calculation of the optimal update vector. Moreover, the proposed scheme fosters multi-task optimization by establishing a common global flat minima for all the concerned tasks. This attribute renders it particularly fitting for scenarios such as object detection, inherently necessitating the joint optimization of both classification and localization. Comprehensive experiments on the PASCAL VOC and MS COCO datasets have shown that BNN-SAM can easily improve the performance of a baseline binary SSD300 detector to outperform state-of-the-art binary detectors, including BiDet, AutoBiDet and LWS-Det, by 6.5%, 5.0%, 1.1%, respectively, without the need of extra optimization approaches. Code is available at https://github.com/Anonymous2740/BNN-SAM.

Joint-Guided Distillation Binary Neural Network Via Dynamic Channel-Wise Diversity Enhancement for Object Detection

An Information-Reserved and Deviation-Controllable Binary Neural Network for Object Detection

Mixed-precision Quantization with Dynamical Hessian Matrix for Object Detection Network

Df-net: diversity-focused network for video object detection

Joint Loss-Based Multi-decoder Network for OCT Fluid Segmentation

Mixed-Precision Quantized Neural Network with Progressively Decreasing Bitwidth For Image Classification and Object Detection.

BOB-YOLO: Balancing Optimization Binarized YOLO Via Module-Wise Latency

Joint-DetNAS: Upgrade Your Detector with NAS, Pruning and Dynamic Distillation

Learning Efficient Binarized Object Detectors With Information Compression

Fully Quantized Network For Object Detection

QD-BEV : Quantization-aware View-guided Distillation for Multi-view 3D Object Detection

BNN-SAM: Improving generalization of binary object detector by Seeking Flat Minima

LC-BiDet: Laterally Connected Binary Detector with Efficient Image Processing

Mixed-precision Deep Neural Network Quantization With Multiple Compression Rates

Group channel pruning and spatial attention distilling for object detection

REQ-YOLO

DPQ: dynamic pseudo-mean mixed-precision quantization for pruned neural network

Model-Driven Deep Learning for Distributed Detection with Binary Quantization

GradQuant: Low-Loss Quantization for Remote-Sensing Object Detection

AQD: Towards Accurate Quantized Object Detection

Joint Denoising and Demosaicking With Green Channel Prior for Real-World Burst Images