Abstract: Deep neural networks (DNNs) have become a key part of many modern software applications. After training and validating, the DNN is deployed as an irrevocable component and applied in real-world scenarios. Although most DNNs are built meticulously with huge volumes of training data, data in real-world still remain unknown to the DNN model, which leads to the crucial requirement of runtime out-of-distribution (OOD) detection. However, many existing approaches 1) need OOD data for classifier training or parameter tuning, or 2) simply combine the scores of each hidden layer as an ensemble of features for OOD detection. In this paper, we present a novel outlook on in-distribution data in a generative manner, which takes their latent features generated from each hidden layer as a joint distribution across representation spaces. Since only the in-distribution latent features are comprehensively understood in representation space, the internal difference between in-distribution and OOD data can be naturally revealed without the intervention of any OOD data. Specifically, We construct a generative model, called Latent Sequential Gaussian Mixture (LSGM), to depict how the in-distribution latent features are generated in terms of the trace of DNN inference across representation spaces. We first construct the Gaussian Mixture Model (GMM) based on in-distribution latent features for each hidden layer, and then connect GMMs via the transition probabilities of the inference traces. Experimental evaluations on popular benchmark OOD datasets and models validate the superiority of the proposed method over the state-of-the-art methods in OOD detection.

Dimensionality-induced information loss of outliers in deep neural networks

Detecting Out-of-Distribution Through the Lens of Neural Collapse

A statistical framework for efficient out of distribution detection in deep neural networks

Understanding the Feature Norm for Out-of-Distribution Detection

Low-Dimensional Gradient Helps Out-of-Distribution Detection

Out-of-Distribution Detection for Deep Neural Networks with Isolation Forest and Local Outlier Factor

Are all outliers alike? On Understanding the Diversity of Outliers for Detecting OODs

Towards In-Distribution Compatible Out-of-Distribution Detection.

An Empirical Study on Robustness of DNNs with Out-of-Distribution Awareness.

Density-driven Regularization for Out-of-distribution Detection

Uncertainty-Aware Out-of-Distribution Detection with Gaussian Processes

Rethinking Out-of-Distribution Detection on Imbalanced Data Distribution

Out-of-Distribution Detection with Deep Nearest Neighbors

A Novel Statistical Measure for Out-of-Distribution Detection in Data Quality Assurance

Out-of-distribution detection based on subspace projection of high-dimensional features output by the last convolutional layer

Detecting Out-of-Distribution Examples Via Class-Conditional Impressions Reappearing

Hyperdimensional Feature Fusion for Out-Of-Distribution Detection

Feature Space Singularity for Out-of-Distribution Detection

Out-of-Distribution Detection with Virtual Outlier Smoothing

A Simple Test-Time Method for Out-of-Distribution Detection

Joint Distribution across Representation Space for Out-of-Distribution Detection