Abstract:Detecting out-of-distribution (OOD) samples poses a significant safety challenge when deploying models in open-world scenarios. Advanced works assume that OOD and in-distributional (ID) samples exhibit a distribution discrepancy, showing an encouraging direction in estimating the uncertainty with embedding features or predicting outputs. Besides incorporating auxiliary outlier as decision boundary, quantifying a “meaningful distance” in embedding space as uncertainty measurement is a promising strategy. However, these distances-based approaches overlook the data structure and heavily rely on the high-dimension features learned by deep neural networks, causing unreliable distances due to the “curse of dimensionality”. In this work, we propose a data structure-aware approach to mitigate the sensitivity of distances to the “curse of dimensionality”, where high-dimensional features are mapped to the manifold of ID samples, leveraging the well-known manifold assumption. Specifically, we present a novel distance termed as tangent distance, which tackles the issue of generalizing the meaningfulness of distances on testing samples to detect OOD inputs. Inspired by manifold learning for adversarial examples, where adversarial region probability density is close to the orthogonal direction of the manifold, and both OOD and adversarial samples have common characteristic−imperceptible perturbations with shift distribution, we propose that OOD samples are relatively far away from the ID manifold, where tangent distance directly computes the Euclidean distance between samples and the nearest submanifold space−instantiated as the linear approximation of local region on the manifold. We provide empirical and theoretical insights to demonstrate the effectiveness of OOD uncertainty measurements on the low-dimensional subspace. Extensive experiments show that the tangent distance performs competitively with other post hoc OOD detection baselines on common and large-scale benchmarks, and the theoretical analysis supports our claim that ID samples are likely to reside in high-density regions, explaining the effectiveness of internal connections among ID data

Dimensionality-reduced Secure Outlier Detection on Union of Subspaces

Provable Self-Representation Based Outlier Detection in a Union of Subspaces

Robust Subspace Outlier Detection in High Dimensional Space

SUOD: Toward Scalable Unsupervised Outlier Detection

A Self-Representation Method with Local Similarity Preserving for Fast Multi-View Outlier Detection

Characterizing Submanifold Region for Out-of-Distribution Detection

E 3 Outlier : A Self-Supervised Framework for Unsupervised Deep Outlier Detection

Cascade Subspace Clustering for Outlier Detection

Ordinal Outlier Detection Based On Recursive Uniform Partitioning

Sparse random projection isolation forest for outlier detection

Exploring feature sparsity for out-of-distribution detection

Feature Space Singularity for Out-of-Distribution Detection.

Feature Space Singularity for Out-of-Distribution Detection

Human-in-the-loop Outlier Detection.

Detecting outliers by clustering algorithms

NDOD: an Efficient Neighboring Dependent Outlier Detector for Bias Distributed Large Datasets

Dimensionality-Aware Outlier Detection: Theoretical and Experimental Analysis

Unsupervised Outlier Detection using Random Subspace and Subsampling Ensembles of Dirichlet Process Mixtures

DOS: Diverse Outlier Sampling for Out-of-Distribution Detection

Diffusion based Semantic Outlier Generation via Nuisance Awareness for Out-of-Distribution Detection

A Novel Outlier Detection Method for Multivariate Data