Abstract:Autonomous vehicles require a precise understanding of their environment to navigate safely. Reliable identification of unknown objects, especially those that are absent during training, such as wild animals, is critical due to their potential to cause serious accidents. Significant progress in semantic segmentation of anomalies has been driven by the availability of out-of-distribution (OOD) benchmarks. However, a comprehensive understanding of scene dynamics requires the segmentation of individual objects, and thus the segmentation of instances is essential. Development in this area has been lagging, largely due to the lack of dedicated benchmarks. To address this gap, we have extended the most commonly used anomaly segmentation benchmarks to include the instance segmentation task. Our evaluation of anomaly instance segmentation methods shows that this challenge remains an unsolved problem. The benchmark website and the competition page can be found at: <a class="link-external link-https" href="https://vision.rwth-aachen.de/oodis" rel="external noopener nofollow">this https URL</a> .

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: **How to accurately detect and segment unseen out - of - distribution (OOD) objects in the autonomous driving scenario**. Specifically, the paper focuses on the instance segmentation problem of abnormal objects (such as wild animals, dropped cargo, etc.) that have not appeared in the training data. ### Problem Background 1. **Requirements for Autonomous Driving**: Autonomous vehicles need to have an accurate understanding of the surrounding environment to ensure safe navigation. Especially for those abnormal objects (such as wild deer, dropped boxes, etc.) that have not been seen during training, reliable identification is crucial because these objects may cause serious traffic accidents. 2. **Limitations of Existing Benchmarks**: Although existing abnormal segmentation benchmarks (such as Fishyscapes, RoadAnomaly21, etc.) have made significant progress in semantic segmentation, they mainly focus on pixel - level segmentation and lack the ability to perform instance segmentation on individual abnormal objects. This makes it difficult for the model to distinguish between multiple abnormal objects, especially in complex scenes. 3. **Importance of Instance Segmentation**: In order to better understand the dynamic changes in the scene, especially when there are multiple abnormal objects in the scene (such as a herd of cows or sheep), instance segmentation is very important. It can help the system more accurately understand the behavior of each object, thereby improving downstream tasks (such as navigation and planning). ### Main Contributions of the Paper 1. **Proposing a New Benchmark (OoDIS)**: To solve the above problems, the authors extended the existing abnormal segmentation benchmarks and introduced the instance segmentation task. They re - annotated three popular abnormal segmentation datasets (Fishyscapes, RoadAnomaly21, and RoadObstacle21) and provided a unified evaluation framework. 2. **Evaluating Existing Methods**: The authors used the newly proposed benchmark to evaluate existing abnormal instance segmentation methods and found that current methods perform poorly when dealing with distant and small abnormal objects, especially when generating accurate segmentation masks. 3. **Promoting Research Progress**: By providing a public benchmark and evaluation platform, the authors hope to encourage more researchers to focus on this area and develop more powerful abnormal instance segmentation methods, ultimately improving the safety of autonomous driving systems. ### Summary This paper aims to fill the gap in the field of abnormal instance segmentation, especially in the autonomous driving scenario. Through the extension of existing benchmarks and the evaluation of new methods, it promotes the further development of this field.

OoDIS: Anomaly Instance Segmentation Benchmark

SegmentMeIfYouCan: A Benchmark for Anomaly Segmentation

UGainS: Uncertainty Guided Anomaly Instance Segmentation

AnoVox: A Benchmark for Multimodal Anomaly Detection in Autonomous Driving

WildOcc: A Benchmark for Off-Road 3D Semantic Occupancy Prediction

nnOOD: A Framework for Benchmarking Self-supervised Anomaly Localisation Methods

A Comprehensive Library for Benchmarking Multi-class Visual Anomaly Detection

IDD-AW: A Benchmark for Safe and Robust Segmentation of Drive Scenes in Unstructured Traffic and Adverse Weather

UMAD: University of Macau Anomaly Detection Benchmark Dataset

On the Potential of Open-Vocabulary Models for Object Detection in Unusual Street Scenes

Detection and Retrieval of Out-of-Distribution Objects in Semantic Segmentation

IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing

BLVD: Building A Large-scale 5D Semantics Benchmark for Autonomous Driving

Anomaly-Aware Semantic Segmentation via Style-Aligned OoD Augmentation

MOOD 2020: A public Benchmark for Out-of-Distribution Detection and Localization on medical Images

OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detection

An Efficient Instance Segmentation Framework Using Segmentation Foundation Models with Oriented Bounding Box Prompts

Occluded Video Instance Segmentation: A Benchmark

Diffusion for Out-of-Distribution Detection on Road Scenes and Beyond

Align and Distill: Unifying and Improving Domain Adaptive Object Detection