Abstract:Deep learning models excel when the data distribution during training aligns with testing data. Yet, their performance diminishes when faced with out-of-distribution (OOD) samples, leading to great interest in the field of OOD detection. Current approaches typically assume that OOD samples originate from an unconcentrated distribution complementary to the training distribution. While this assumption is appropriate in the traditional unsupervised OOD (U-OOD) setting, it proves inadequate when considering the place of deployment of the underlying deep learning model. To better reflect this real-world scenario, we introduce the novel setting of continual U-OOD detection. To tackle this new setting, we propose a method that starts from a U-OOD detector, which is agnostic to the OOD distribution, and slowly updates during deployment to account for the actual OOD distribution. Our method uses a new U-OOD scoring function that combines the Mahalanobis distance with a nearest-neighbor approach. Furthermore, we design a confidence-scaled few-shot OOD detector that outperforms previous methods. We show our method greatly improves upon strong baselines from related fields.

What problem does this paper attempt to address?

The paper primarily addresses the problem of continuous unsupervised Out-of-Distribution (OOD) detection. ### Research Background and Objectives - **Research Background**: Deep learning models perform well when the training data distribution is consistent with the test data distribution, but their performance degrades when faced with OOD samples. Existing OOD detection methods usually assume that OOD samples come from a distribution that is unrelated to the training data and is relatively uniform. This assumption is reasonable in traditional unsupervised OOD (U-OOD) settings but is not always applicable in real-world deployment scenarios. - **Research Objectives**: The paper proposes a new setting—continuous U-OOD detection, aiming to better reflect real-world application scenarios. Specifically, this setting considers the actual OOD samples encountered by the model during deployment and gradually updates the model to adapt to these samples. The goal is to develop a method that starts with a U-OOD detector insensitive to OOD distribution and slowly updates during deployment to account for the actual OOD distribution. ### Main Contributions 1. **New Setting and Evaluation Criteria**: Introduced the problem of continuous U-OOD detection and proposed new evaluation metrics and benchmarks to assess the performance of methods under this setting. 2. **Improved Detector**: Proposed a few-shot OOD detector based on confidence scaling, which combines Mahalanobis distance and nearest neighbor methods to score OOD samples. 3. **Innovative Method**: Proposed a new method for continuous U-OOD detection that achieves better results on strong baseline methods in the relevant field. ### Experiments and Validation - **Experimental Setup**: Conducted experimental validation through three different benchmark datasets, including multi-class, single-class, and high-resolution image datasets. - **Evaluation Metrics**: In addition to traditional FPR@95 and AUC metrics, two new metrics—AUF (Area Under the FPR@95 Curve) and AUA (Area Under the AUC Curve)—were proposed to evaluate the model's performance over time. - **Results Analysis**: Experimental results show that the proposed continuous U-OOD detection method outperforms existing methods on multiple benchmark datasets, especially in handling a small number of OOD samples. This indicates that the method can effectively utilize a small number of real OOD samples for self-iterative improvement, thereby enhancing overall OOD detection performance. In summary, the paper proposes an effective solution to the challenge of continuous OOD detection in practical applications and demonstrates its effectiveness through experiments.

Continual Unsupervised Out-of-Distribution Detection

Data Invariants to Understand Unsupervised Out-of-Distribution Detection

Mahalanobis-Aware Training for Out-of-Distribution Detection

Out-of-Distribution Detection with Deep Nearest Neighbors

Out-of-Distribution (OOD) Detection Based on Deep Learning: A Review

Resultant: Incremental Effectiveness on Likelihood for Unsupervised Out-of-Distribution Detection

Out-of-Distribution Detection for LiDAR-based 3D Object Detection

Out-of-Distribution Learning with Human Feedback

Learning to Augment Distributions for Out-of-Distribution Detection

A statistical framework for efficient out of distribution detection in deep neural networks

Meta OOD Learning for Continuously Adaptive OOD Detection

Toward a Realistic Benchmark for Out-of-Distribution Detection

Going Beyond Conventional OOD Detection

Density-driven Regularization for Out-of-distribution Detection

Unveiling the unseen: novel strategies for object detection beyond known distributions

Your Data Is Not Perfect: Towards Cross-Domain Out-of-Distribution Detection in Class-Imbalanced Data

Learning Non-Linear Invariants for Unsupervised Out-of-Distribution Detection

Non-Linear Outlier Synthesis for Out-of-Distribution Detection

A Metacognitive Approach to Out-of-Distribution Detection for Segmentation

A Survey on Out-of-Distribution Detection in NLP

Towards In-Distribution Compatible Out-of-Distribution Detection.