Abstract:We propose a new setting for detecting unseen objects called Zero-shot Annotation object Detection (ZAD). It expands the zero-shot object detection setting by allowing the novel objects to exist in the training images and restricts the additional information the detector uses to novel category names. Recently, to detect unseen objects, large-scale vision-language models (e.g., CLIP) are leveraged by different methods. The distillation-based methods have good overall performance but suffer from a long training schedule caused by two factors. First, existing work creates distillation regions biased to the base categories, which limits the distillation of novel category information. Second, directly using the raw feature from CLIP for distillation neglects the domain gap between the training data of CLIP and the detection datasets, which makes it difficult to learn the mapping from the image region to the vision-language feature space. To solve these problems, we propose Efficient feature distillation for Zero-shot Annotation object Detection (EZAD). Firstly, EZAD adapts the CLIP's feature space to the target detection domain by re-normalizing CLIP; Secondly, EZAD uses CLIP to generate distillation proposals with potential novel category names to avoid the distillation being overly biased toward the base categories. Finally, EZAD takes advantage of semantic meaning for regression to further improve the model performance. As a result, EZAD outperforms the previous distillation-based methods in COCO by 4% with a much shorter training schedule and achieves a 3% improvement on the LVIS dataset. Our code is available at <a class="link-external link-https" href="https://github.com/dragonlzm/EZAD" rel="external noopener nofollow">this https URL</a>

Zero-Annotation Object Detection with Web Knowledge Transfer

Zero-Shot Detection with Transferable Object Proposal Mechanism.

Joint Feature-Level And Pixel-Level Domain Adaption For Object Detection In The Wild

Research on Knowledge Distillation Algorithm of Object Detection

Domain adaptive object detection with model-agnostic knowledge transferring

KTCN: Enhancing Open-World Object Detection with Knowledge Transfer and Class-Awareness Neutralization

Multilevel Knowledge Transfer for Cross-Domain Object Detection

Exploiting Web Images for Weakly Supervised Object Detection

Zero-Shot Classification by Transferring Knowledge and Preserving Data Structure

Knowledge Transfer for Object Detection with Evolution Architecture Search

Semantic Consistency Knowledge Transfer for Unsupervised Cross Domain Object Detection

Cross-domain Constrained Network for Zero-shot Object Detection

Cross-domain Adaptive Object Detection Based on Refined Knowledge Transfer and Mined Guidance in Autonomous Vehicles

Efficient Feature Distillation for Zero-shot Annotation Object Detection

Large Scale Semi-Supervised Object Detection Using Visual and Semantic Knowledge Transfer

Multi-source unsupervised domain adaptation for object detection

Multidomain Object Detection Framework Using Feature Domain Knowledge Distillation.

Unifying Unsupervised Domain Adaptation and Zero-Shot Visual Recognition

Object Detection Based on Semi-Supervised Domain Adaptation for Imbalanced Domain Resources.

Visual and Semantic Knowledge Transfer for Large Scale Semi-supervised Object Detection.

An Introduction to Domain Adaptive Object Detection from Synthesis to Reality