Abstract:OmDet, a novel language‐aware detector, designed to enhance open‐vocabulary and open‐world object detection through a continual learning approach and multi‐dataset vision‐language pre‐training is presented. By using natural language for knowledge representation, the authors successfully increase the "visual vocabularyˮ; and create a unified, language‐conditioned detection framework, which outperforms previous models on object detection and phrase grounding. This promising method proves the effectiveness of joint learning from multiple datasets and presents a path forward for scaling to even larger datasets. The advancement of object detection (OD) in open‐vocabulary and open‐world scenarios is a critical challenge in computer vision. OmDet, a novel language‐aware object detection architecture and an innovative training mechanism that harnesses continual learning and multi‐dataset vision‐language pre‐training is introduced. Leveraging natural language as a universal knowledge representation, OmDet accumulates "visual vocabularies" from diverse datasets, unifying the task as a language‐conditioned detection framework. The multimodal detection network (MDN) overcomes the challenges of multi‐dataset joint training and generalizes to numerous training datasets without manual label taxonomy merging. The authors demonstrate superior performance of OmDet over strong baselines in object detection in the wild, open‐vocabulary detection, and phrase grounding, achieving state‐of‐the‐art results. Ablation studies reveal the impact of scaling the pre‐training visual vocabulary, indicating a promising direction for further expansion to larger datasets. The effectiveness of our deep fusion approach is underscored by its ability to learn jointly from multiple datasets, enhancing performance through knowledge sharing.

BigDetection: A Large-scale Benchmark for Improved Object Detector Pre-training

Universal Object Detection with Large Vision Model

Towards Large-Scale Small Object Detection: Survey and Benchmarks.

SODA10M: Towards Large-Scale Object Detection Benchmark for Autonomous Driving

Plain-Det: A Plain Multi-Dataset Object Detector

A Comprehensive Study on Upper-Body Detection with Deep Neural Networks

Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark

A Simple Approach and Benchmark for 21, 000-Category Object Detection.

OmDet: Large‐scale vision‐language multi‐dataset pre‐training with multimodal detection network

Sampling Techniques for Large-Scale Object Detection From Sparsely Annotated Objects

How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detection

Multi-attribute Object Detection Benchmark for Smart City.

Large Scale Semi-Supervised Object Detection Using Visual and Semantic Knowledge Transfer

Detection Beyond What and Where: A Benchmark for Detecting Occlusion State.

Large Batch Optimization for Object Detection: Training COCO in 12 minutes

CDTD: A Large-Scale Cross-Domain Benchmark for Instance-Level Image-to-Image Translation and Domain Adaptive Object Detection.

3D Object Detection on Large-Scale Dataset

Bag of Freebies for Training Object Detection Neural Networks.

ScaleDet: A Scalable Multi-Dataset Object Detector

MMDetection: Open MMLab Detection Toolbox and Benchmark.

Simple Multi-dataset Detection