Abstract:Zero-shot detection (ZSD) aims to locate and classify unseen objects in pictures or videos by semantic auxiliary information without additional training examples. Most of the existing ZSD methods are based on two-stage models, which achieve the detection of unseen classes by aligning object region proposals with semantic embeddings. However, these methods have several limitations, including poor region proposals for unseen classes, lack of consideration of semantic representations of unseen classes or their inter-class correlations, and domain bias towards seen classes, which can degrade overall performance. To address these issues, the Trans-ZSD framework is proposed, which is a transformer-based multi-scale contextual detection framework that explicitly exploits inter-class correlations between seen and unseen classes and optimizes feature distribution to learn discriminative features. Trans-ZSD is a single-stage approach that skips proposal generation and performs detection directly, allowing the encoding of long-term dependencies at multiple scales to learn contextual features while requiring fewer inductive biases. Trans-ZSD also introduces a foreground-background separation branch to alleviate the confusion of unseen classes and backgrounds, contrastive learning to learn inter-class uniqueness and reduce misclassification between similar classes, and explicit inter-class commonality learning to facilitate generalization between related classes. Trans-ZSD addresses the domain bias problem in end-to-end generalized zero-shot detection (GZSD) models by using balance loss to maximize response consistency between seen and unseen predictions, ensuring that the model does not bias towards seen classes. The Trans-ZSD framework is evaluated on the PASCAL VOC and MS COCO datasets, demonstrating significant improvements over existing ZSD models.

Experiments on Zero-Shot Object Detection for Indoor Robots

Zero-Shot Detection with Transferable Object Proposal Mechanism.

Context-Guided Super-Class Inference for Zero-Shot Detection

Transformer-Based Approach Via Contrastive Learning for Zero-Shot Detection.

A Survey of Zero Shot Detection: Methods and Applications

Cross-domain Constrained Network for Zero-shot Object Detection

Zero-Shot Object Detection by Hybrid Region Embedding

Zero-Shot Object Detection by Semantics-Aware DETR with Adaptive Contrastive Loss.

Robust Region Feature Synthesizer for Zero-Shot Object Detection

Learning Latent Semantic Attributes for Zero-Shot Object Detection.

Extracting Zero-shot Common Sense from Large Language Models for Robot 3D Scene Understanding

ZeroSCD: Zero-Shot Street Scene Change Detection

Zero-Shot Recognition Based on Semantic Embeddings and Deep Clustering

Zero-Shot Recognition Using Dual Visual-Semantic Mapping Paths.

Recent Advances in Zero-shot Recognition

Semantics-Preserving Graph Propagation for Zero-Shot Object Detection

Frustratingly Simple but Effective Zero-shot Detection and Segmentation: Analysis and a Strong Baseline

Zero-Shot Co-salient Object Detection Framework

Incrementally Zero-Shot Detection by an Extreme Value Analyzer

Boosting Object Detection with Zero-Shot Day-Night Domain Adaptation

ZeroPose: CAD-Prompted Zero-shot Object 6D Pose Estimation in Cluttered Scenes