Abstract:Unsupervised Outlier Detection (UOD) is an important data mining task. With the advance of deep learning, deep Outlier Detection (OD) has received broad interest. Most deep UOD models are trained exclusively on clean datasets to learn the distribution of the normal data, which requires huge manual efforts to clean the real-world data if possible. Instead of relying on clean datasets, some approaches directly train and detect on unlabeled contaminated datasets, leading to the need for methods that are robust to such conditions. Ensemble methods emerged as a superior solution to enhance model robustness against contaminated training sets. However, the training time is greatly increased by the ensemble. In this study, we investigate the impact of outliers on the training phase, aiming to halt training on unlabeled contaminated datasets before performance degradation. Initially, we noted that blending normal and anomalous data causes AUC fluctuations, a label-dependent measure of detection accuracy. To circumvent the need for labels, we propose a zero-label entropy metric named Loss Entropy for loss distribution, enabling us to infer optimal stopping points for training without labels. Meanwhile, we theoretically demonstrate negative correlation between entropy metric and the label-based AUC. Based on this, we develop an automated early-stopping algorithm, EntropyStop, which halts training when loss entropy suggests the maximum model detection capability. We conduct extensive experiments on ADBench (including 47 real datasets), and the overall results indicate that AutoEncoder (AE) enhanced by our approach not only achieves better performance than ensemble AEs but also requires under 2\% of training time. Lastly, our proposed metric and early-stopping approach are evaluated on other deep OD models, exhibiting their broad potential applicability.

Out-of-Distribution Detection using Maximum Entropy Coding

Detecting Out-of-Distribution Samples via Conditional Distribution Entropy with Optimal Transport

Out-of-distribution detection for regression tasks: parameter versus predictor entropy

Learning by Erasing: Conditional Entropy based Transferable Out-Of-Distribution Detection

Out-of-Distribution Detection using BiGAN and MDL

Learning to Embed Distributions via Maximum Kernel Entropy

Optimizing Distributions for Associated Entropic Vectors via Generative Convolutional Neural Networks

Out-of-Distribution Detection with Overlap Index

Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection

Maximum entropy based testing for fuzzy exponential random variable and its applications

Gaussian Process Regression for Maximum Entropy Distribution

Resultant: Incremental Effectiveness on Likelihood for Unsupervised Out-of-Distribution Detection

MaxEnt Loss: Constrained Maximum Entropy for Calibration under Out-of-Distribution Shift

Out-of-Distribution Detection with Hilbert-Schmidt Independence Optimization

Certifiable Out-of-Distribution Generalization.

A family of discrete maximum-entropy distributions

Out-of-Distribution Detection with a Single Unconditional Diffusion Model

EntropyStop: Unsupervised Deep Outlier Detection with Loss Entropy

A View on Out-of-Distribution Identification from a Statistical Testing Theory Perspective

Robust Discriminant Analysis Based on Nonparametric Maximum Entropy