Abstract:Out-of-distribution detection (OOD) is a crucial technique for deploying machine learning models in the real world to handle the unseen scenarios. In this paper, we first propose a simple yet effective Neural Activation Prior (NAP) for OOD detection. Our neural activation prior is based on a key observation that, for a channel before the global pooling layer of a fully trained neural network, the probability of a few neurons being activated with a large response by an in-distribution (ID) sample is significantly higher than that by an OOD sample. An intuitive explanation is that for a model fully trained on ID dataset, each channel would play a role in detecting a certain pattern in the ID dataset, and a few neurons can be activated with a large response when the pattern is detected in an input sample. Then, a new scoring function based on this prior is proposed to highlight the role of these strongly activated neurons in OOD detection. Our approach is plug-and-play and does not lead to any performance degradation on ID data classification and requires no extra training or statistics from training or external datasets. Notice that previous methods primarily rely on post-global-pooling features of the neural networks, while the within-channel distribution information we leverage would be discarded by the global pooling operator. Consequently, our method is orthogonal to existing approaches and can be effectively combined with them in various applications. Experimental results show that our method achieves the state-of-the-art performance on CIFAR benchmark and ImageNet dataset, which demonstrates the power of the proposed prior. Finally, we extend our method to Transformers and the experimental findings indicate that NAP can also significantly enhance the performance of OOD detection on Transformers, thereby demonstrating the broad applicability of this prior knowledge.

Out-of-Distribution Detection Through Relative Activation-Deactivation Abstractions

Towards In-Distribution Compatible Out-of-Distribution Detection.

Out-of-Distribution Detection & Applications With Ablated Learned Temperature Energy

Mahalanobis-Aware Training for Out-of-Distribution Detection

The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection

Advancing Out-of-Distribution Detection through Data Purification and Dynamic Activation Function Design

Rethinking Out-of-Distribution Detection From a Human-Centric Perspective

Out-of-Distribution Detection with Deep Nearest Neighbors

Out-of-Distribution (OOD) Detection Based on Deep Learning: A Review

Out-of-Distribution Detection by Cross-Class Vicinity Distribution of In-Distribution Data

Density-driven Regularization for Out-of-distribution Detection

Unleashing Mask: Explore the Intrinsic Out-of-Distribution Detection Capability

Re-Benchmarking Out-of-Distribution Detection in Deep Neural Networks

Out-of-Distribution Detection using Neural Activation Prior

Learning to Augment Distributions for Out-of-Distribution Detection

Learning by Erasing: Conditional Entropy based Transferable Out-Of-Distribution Detection

Class Relevance Learning For Out-of-distribution Detection

Hierarchical Visual Categories Modeling: A Joint Representation Learning and Density Estimation Framework for Out-of-Distribution Detection

Recent Advances in OOD Detection: Problems and Approaches

Hypothesis-Driven Deep Learning for Out of Distribution Detection

Extremely Simple Activation Shaping for Out-of-Distribution Detection