Abstract:Object detection is a pivotal task in computer vision that has received significant attention in previous years. Nonetheless, the capability of a detector to localise objects out of the training distribution remains unexplored. Whilst recent approaches in object-level out-of-distribution (OoD) detection heavily rely on class labels, such approaches contradict truly open-world scenarios where the class distribution is often unknown. In this context, anomaly detection focuses on detecting unseen instances rather than classifying detections as OoD. This work aims to bridge this gap by leveraging an open-world object detector and an OoD detector via virtual outlier synthesis. This is achieved by using the detector backbone features to first learn object pseudo-classes via self-supervision. These pseudo-classes serve as the basis for class-conditional virtual outlier sampling of anomalous features that are classified by an OoD head. Our approach empowers our overall object detector architecture to learn anomaly-aware feature representations without relying on class labels, hence enabling truly open-world object anomaly detection. Empirical validation of our approach demonstrates its effectiveness across diverse datasets encompassing various imaging modalities (visible, infrared, and X-ray). Moreover, our method establishes state-of-the-art performance on object-level anomaly detection, achieving an average recall score improvement of over 5.4% for natural images and 23.5% for a security X-ray dataset compared to the current approaches. In addition, our method detects anomalies in datasets where current approaches fail. Code available at <a class="link-external link-https" href="https://github.com/KostadinovShalon/oln-ssos" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The paper aims to address the problem of open-world object anomaly detection in the field of computer vision, particularly in the detection and recognition of unknown category objects. Specifically, the research goal is to effectively detect objects in images that do not belong to the training data distribution (i.e., anomalous or unknown category objects) without explicit category labels. To achieve this goal, the authors propose a novel framework that combines an Open World Object Detector (OWOD) with an anomaly detection module based on Virtual Outlier Synthesis (VOS). The framework learns pseudo-categories of objects in a self-supervised manner and uses these pseudo-categories to generate virtual outlier samples, thereby training a classifier capable of distinguishing between normal and anomalous objects. This approach enables the model to learn to recognize anomalous objects without relying on specific category labels. The main contributions of the paper include: 1. Proposing the first end-to-end, category-agnostic object-level anomaly detection architecture that learns object pseudo-categories in a self-supervised manner and fits a conditional Gaussian distribution in the object feature space of the open-world object detector, thereby achieving energy-based anomaly detection. 2. Demonstrating state-of-the-art anomaly detection performance on multiple benchmark datasets, including datasets with various imaging modalities such as natural images and X-ray images. 3. Achieving effective identification of anomalous instances that previous methods failed to detect, and proving through qualitative analysis that the model can simultaneously locate unknown objects in images and classify them as anomalies. 4. Validating the impact of design choices in the method through ablation studies, such as the number of clusters and the use of instance masks. In summary, the proposed method in the paper addresses one of the key challenges in current anomaly detection, which is how to effectively detect anomalous objects in images in the absence of explicit category information.

Towards Open-World Object-based Anomaly Detection via Self-Supervised Outlier Synthesis

VOS: Learning What You Don't Know by Virtual Outlier Synthesis

Object-Oriented Anomaly Detection In Surveillance Videos

nnOOD: A Framework for Benchmarking Self-supervised Anomaly Localisation Methods

Confidence-Aware and Self-Supervised Image Anomaly Localisation

On the Potential of Open-Vocabulary Models for Object Detection in Unusual Street Scenes

Exploring Orthogonality in Open World Object Detection

Non-Linear Outlier Synthesis for Out-of-Distribution Detection

Unveiling the unseen: novel strategies for object detection beyond known distributions

View-Invariant Pixelwise Anomaly Detection in Multi-object Scenes with Adaptive View Synthesis

Finding Dino: A plug-and-play framework for unsupervised detection of out-of-distribution objects using prototypes

Synthetic outlier generation for anomaly detection in autonomous driving

Open-Set 3D object detection in LiDAR data as an Out-of-Distribution problem

Online Open-set Semi-supervised Object Detection by Valuable Instances Mining

Going Beyond Conventional OOD Detection

Visual Out-of-Distribution Detection in Open-Set Noisy Environments

Open-Vocabulary Video Anomaly Detection

Anomaly Heterogeneity Learning for Open-set Supervised Anomaly Detection

Natural Synthetic Anomalies for Self-Supervised Anomaly Detection and Localization

Non-Parametric Outlier Synthesis

LS-VOS: Identifying Outliers in 3D Object Detections Using Latent Space Virtual Outlier Synthesis