Abstract:With the emergence of foundation models, deep learning-based object detectors have shown practical usability in closed set scenarios. However, for real-world tasks, object detectors often operate in open environments, where crucial factors (e.g., data distribution, objective) that influence model learning are often changing. The dynamic and intricate nature of the open environment poses novel and formidable challenges to object detectors. Unfortunately, current research on object detectors in open environments lacks a comprehensive analysis of their distinctive characteristics, challenges, and corresponding solutions, which hinders their secure deployment in critical real-world scenarios. This paper aims to bridge this gap by conducting a comprehensive review and analysis of object detectors in open environments. We initially identified limitations of key structural components within the existing detection pipeline and propose the open environment object detector challenge framework that includes four quadrants (i.e., out-of-domain, out-of-category, robust learning, and incremental learning) based on the dimensions of the data / target changes. For each quadrant of challenges in the proposed framework, we present a detailed description and systematic analysis of the overarching goals and core difficulties, systematically review the corresponding solutions, and benchmark their performance over multiple widely adopted datasets. In addition, we engage in a discussion of open problems and potential avenues for future research. This paper aims to provide a fresh, comprehensive, and systematic understanding of the challenges and solutions associated with open-environment object detectors, thus catalyzing the development of more solid applications in real-world scenarios. A project related to this survey can be found at

Towards Evidential and Class Separable Open Set Object Detection

Spatial Likelihood Voting with Self-Knowledge Distillation for Weakly Supervised Object Detection.

Wasserstein Distance-based Expansion of Low-Density Latent Regions for Unknown Class Detection

A Dual-threshold Based Evidential Openmax Approach for Open Set Recognition

Adaptive Negative Evidential Deep Learning for Open-set Semi-supervised Learning

Object Detectors in the Open Environment: Challenges, Solutions, and Outlook

Open-Set Object Detection Using Classification-free Object Proposal and Instance-level Contrastive Learning

UADet: A Remarkably Simple Yet Effective Uncertainty-Aware Open-Set Object Detection Framework

SKDF: A Simple Knowledge Distillation Framework for Distilling Open-Vocabulary Knowledge to Open-world Object Detector

Few-Shot Open-Set Object Detection via Conditional Evidence Decoupling

USD: Unknown Sensitive Detector Empowered by Decoupled Objectness and Segment Anything Model

Enhancing Open-Set Object Detection Via Uncertainty-Boxes Identification.

Towards Open-set Camera 3D Object Detection

Unsupervised Recognition of Unknown Objects for Open-World Object Detection

Open-World Object Detection with Instance Representation Learning

Learning Clear Class Separation for Open-set 3D Detector in Autonomous Vehicle via Selective Forgetting

Open-Set 3D Detection via Image-level Class and Debiased Cross-modal Contrastive Learning

YOLOOC: YOLO-based Open-Class Incremental Object Detection with Novel Class Discovery

DDOWOD: DiffusionDet for Open-World Object Detection

On the Potential of Open-Vocabulary Models for Object Detection in Unusual Street Scenes

Evidential Deep Learning for Open Set Action Recognition