Abstract:Deep learning systems deployed in real-world applications often encounter data that is different from their in-distribution (ID). A reliable system should ideally abstain from making decisions in this out-of-distribution (OOD) setting. Existing state-of-the-art methods primarily focus on feature distances, such as k-th nearest neighbors and distances to decision boundaries, either overlooking or ineffectively using in-distribution statistics. In this work, we propose a novel angle-based metric for OOD detection that is computed relative to the in-distribution structure. We demonstrate that the angles between feature representations and decision boundaries, viewed from the mean of in-distribution features, serve as an effective discriminative factor between ID and OOD data. Our method achieves state-of-the-art performance on CIFAR-10 and ImageNet benchmarks, reducing FPR95 by 0.88% and 7.74% respectively. Our score function is compatible with existing feature space regularization techniques, enhancing performance. Additionally, its scale-invariance property enables creating an ensemble of models for OOD detection via simple score summation.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in practical applications, deep - learning systems often encounter data with a different distribution from the training data (i.e., Out - of - Distribution (OOD) data). A reliable system should be able to recognize and avoid making decisions when encountering these OOD data. The existing state - of - the - art methods mainly focus on feature distances, such as k - nearest neighbors and the distance to the decision boundary, but often ignore or ineffectively use the in - distribution statistics. This paper proposes a new angle - based metric method to detect OOD data, which calculates the angle relative to the in - distribution structure to improve the accuracy of OOD detection. Specifically, the paper calculates the angle between the feature representation and the decision boundary (from the perspective of the mean of the in - distribution features) as an effective discriminant factor for distinguishing between in - distribution and out - distribution data. This method achieves state - of - the - art performance in the CIFAR - 10 and ImageNet benchmarks, reducing the FPR95 metric by 0.88% and 7.74% respectively. In addition, the scale - invariance property of this method allows for the creation of a model ensemble for OOD detection by simply aggregating scores. ### Main contributions of the paper: 1. **Proposed a new OOD detection technique**: Calculate the angle between the feature representation and its projection on the decision boundary, relative to the mean of the in - distribution features. 2. **Model - agnostic, hyperparameter - free and efficient**: This method can be flexibly applied to various architectures without additional parameter tuning. 3. **Demonstrated state - of - the - art performance in the widely - used CIFAR - 10 and ImageNet OOD benchmarks**: In particular, in the large - scale ImageNet OOD benchmark, LAFO reduces the FPR95 score by 7.74% compared to the best previous distance - based method. 4. **The scale - invariance property of LAFO**: Allows for improving the ensemble performance by simply aggregating the scores of multiple pre - trained models. Experiments show that the ensemble using LAFO reduces FPR95 by 2.51% in the ImageNet OOD benchmark. ### Method overview: - **Problem setting**: Consider a supervised classification setting, given a pre - trained model \( f \), the goal is to determine whether an input sample comes from in - distribution data. - **OOD detection based on relative angles**: Distinguish between in - distribution and out - distribution data by calculating the angle between the feature representation and its projection on the decision boundary from the perspective of the mean of the in - distribution features. - **Projection of features on the decision boundary**: Derived the mathematical equation of the decision boundary and showed the properties of the scores. ### Experimental results: - **CIFAR - 10 and ImageNet benchmarks**: LAFO performs excellently in both of these benchmarks, especially on ImageNet, where the FPR95 metric is significantly reduced. - **Model ensemble**: The scale - invariance property of LAFO enables it to improve OOD detection performance by simply aggregating the scores of different models. In conclusion, this paper proposes a novel and effective OOD detection method, which significantly improves the performance of OOD detection by using the relative angle between the feature representation and the decision boundary.

Look Around and Find Out: OOD Detection with Relative Angles

Out-of-Distribution Detection with Deep Nearest Neighbors

Fast Decision Boundary based Out-of-Distribution Detector

Comprehensive OOD Detection Improvements

Improving Out-of-Distribution Detection with Disentangled Foreground and Background Features

Detecting Out-of-Distribution Examples Via Class-Conditional Impressions Reappearing

OAL: Enhancing OOD Detection Using Latent Diffusion

Rethinking Out-of-Distribution Detection on Imbalanced Data Distribution

GradOrth: A Simple yet Efficient Out-of-Distribution Detection with Orthogonal Projection of Gradients

OoDAnalyzer: Interactive Analysis of Out-of-Distribution Samples

Out-of-Distribution Detection by Principal Component Correspondence

Hyperbolic Metric Learning for Visual Outlier Detection

Density-driven Regularization for Out-of-distribution Detection

Toward a Realistic Benchmark for Out-of-Distribution Detection

EPA: Neural Collapse Inspired Robust Out-of-Distribution Detector

From Global to Local: Multi-scale Out-of-distribution Detection

Out-of-Distribution Detection with Overlap Index

Out-of-Distribution Detection with Hilbert-Schmidt Independence Optimization

Calibrated Out-of-Distribution Detection with a Generic Representation

Your Classifier Can Be Secretly a Likelihood-Based OOD Detector

ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithms