Abstract: Deep neural networks (DNNs) have become a key part of many modern software applications. After training and validating, the DNN is deployed as an irrevocable component and applied in real-world scenarios. Although most DNNs are built meticulously with huge volumes of training data, data in real-world still remain unknown to the DNN model, which leads to the crucial requirement of runtime out-of-distribution (OOD) detection. However, many existing approaches 1) need OOD data for classifier training or parameter tuning, or 2) simply combine the scores of each hidden layer as an ensemble of features for OOD detection. In this paper, we present a novel outlook on in-distribution data in a generative manner, which takes their latent features generated from each hidden layer as a joint distribution across representation spaces. Since only the in-distribution latent features are comprehensively understood in representation space, the internal difference between in-distribution and OOD data can be naturally revealed without the intervention of any OOD data. Specifically, We construct a generative model, called Latent Sequential Gaussian Mixture (LSGM), to depict how the in-distribution latent features are generated in terms of the trace of DNN inference across representation spaces. We first construct the Gaussian Mixture Model (GMM) based on in-distribution latent features for each hidden layer, and then connect GMMs via the transition probabilities of the inference traces. Experimental evaluations on popular benchmark OOD datasets and models validate the superiority of the proposed method over the state-of-the-art methods in OOD detection.

Towards Out-of-Distribution Detection with Divergence Guarantee in Deep Generative Models

Kullback-Leibler Divergence-Based Out-of-Distribution Detection with Flow-Based Generative Models

DOI: Divergence-based Out-of-Distribution Indicators via Deep Generative Models

Further Analysis of Outlier Detection with Deep Generative Models.

Out-of-distribution Detection with Diffusion-based Neighborhood

Out-Of-Distribution Detection with Diversification (Provably)

Resultant: Incremental Effectiveness on Likelihood for Unsupervised Out-of-Distribution Detection

Understanding Likelihood of Normalizing Flow and Image Complexity through the Lens of Out-of-Distribution Detection

A Geometric Explanation of the Likelihood OOD Detection Paradox

Towards In-Distribution Compatible Out-of-Distribution Detection.

Joint Distribution across Representation Space for Out-of-Distribution Detection

Model-agnostic out-of-distribution detection using combined statistical tests

Approximations to the Fisher Information Metric of Deep Generative Models for Out-Of-Distribution Detection

Can Your Generative Model Detect Out-of-Distribution Covariate Shift?

Out-of-Distribution Detection with a Single Unconditional Diffusion Model

Exploiting Diffusion Prior for Out-of-Distribution Detection

Uncertainty-Aware Out-of-Distribution Detection with Gaussian Processes

ODIM: Outlier Detection via Likelihood of Under-Fitted Generative Models

Out-of-distribution detection for regression tasks: parameter versus predictor entropy

Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection