Abstract:Binary Neural Network (BNN) exhibits substantial potential for low-cost and energy-efficient deployment, notably in the application of BNN-facilitated object detection on embedded devices. Despite its high compression ratio, the optimization of BNN encounters a significant challenge due to the discrete nature of binary weights and activations. Achieving good generalization performance poses a notable obstacle in this regard. Previous BNN training schemes that only focus on minimizing the empirical loss can easily suffer from overfitting problem, yielding poor generalization capacity when being exposed to unseen data. Sharpness-Aware Minimization (SAM), which simultaneously minimizes the loss value and sharpness of the loss landscape, has emerged as an effective approach for enhancing the generalization ability of neural networks, and has been demonstrated to be effective in improving the performance of BNNs designed for classification tasks. However, their work does not address the optimization challenge of applying SAM to models that handle multiple tasks, such as binary object detection, which involves both classification and location tasks. To address this issue, we propose a modified SAM scheme, denoted as BNN-SAM, which introduces a new objective allowing for the direct calculation of the optimal update vector. Moreover, the proposed scheme fosters multi-task optimization by establishing a common global flat minima for all the concerned tasks. This attribute renders it particularly fitting for scenarios such as object detection, inherently necessitating the joint optimization of both classification and localization. Comprehensive experiments on the PASCAL VOC and MS COCO datasets have shown that BNN-SAM can easily improve the performance of a baseline binary SSD300 detector to outperform state-of-the-art binary detectors, including BiDet, AutoBiDet and LWS-Det, by 6.5%, 5.0%, 1.1%, respectively, without the need of extra optimization approaches. Code is available at https://github.com/Anonymous2740/BNN-SAM.

Binary Similarity Few-Shot Object Detection with Modeling of Hard Negative Samples

SimDet: Cross Similarity Attention for One-shot Object Detection

Multi-Similarity Enhancement Network for Few-Shot Segmentation.

Understanding Negative Proposals in Generic Few-Shot Object Detection

BSNet: Bi-Similarity Network for Few-shot Fine-grained Image Classification

Few-Shot Object Detection with Memory Contrastive Proposal Based on Semantic Priors

Semantic Enhanced Few-shot Object Detection

Bilaterally Normalized Scale-Consistent Sinkhorn Distance for Few-Shot Image Classification

BiDet: An Efficient Binarized Object Detector

Zero-Shot Object Detection by Semantics-Aware DETR with Adaptive Contrastive Loss.

BNN-SAM: Improving generalization of binary object detector by Seeking Flat Minima

Multi-Faceted Distillation of Base-Novel Commonality for Few-shot Object Detection

A Novel Feature-Based Model for Zero-Shot Object Detection with Simulated Attributes

Few-Shot Object Detection with Anti-Confusion Grouping

TA-BiDet: Task-aligned binary object detector

Object-aware Long-short-range Spatial Alignment for Few-Shot Fine-Grained Image Classification

Beyond Max-Margin: Class Margin Equilibrium for Few-shot Object Detection

Mutually Reinforcing Structure with Proposal Contrastive Consistency for Few-Shot Object Detection.

DBDC-SSL: Deep Brownian Distance Covariance With Self-Supervised Learning for Few-Shot Image Classification

Towards Improving Classification Power for One-Shot Object Detection

Your Negative May not Be True Negative: Boosting Image-Text Matching with False Negative Elimination