Abstract:Unsupervised Outlier Detection (UOD) is an important data mining task. With the advance of deep learning, deep Outlier Detection (OD) has received broad interest. Most deep UOD models are trained exclusively on clean datasets to learn the distribution of the normal data, which requires huge manual efforts to clean the real-world data if possible. Instead of relying on clean datasets, some approaches directly train and detect on unlabeled contaminated datasets, leading to the need for methods that are robust to such conditions. Ensemble methods emerged as a superior solution to enhance model robustness against contaminated training sets. However, the training time is greatly increased by the ensemble. In this study, we investigate the impact of outliers on the training phase, aiming to halt training on unlabeled contaminated datasets before performance degradation. Initially, we noted that blending normal and anomalous data causes AUC fluctuations, a label-dependent measure of detection accuracy. To circumvent the need for labels, we propose a zero-label entropy metric named Loss Entropy for loss distribution, enabling us to infer optimal stopping points for training without labels. Meanwhile, we theoretically demonstrate negative correlation between entropy metric and the label-based AUC. Based on this, we develop an automated early-stopping algorithm, EntropyStop, which halts training when loss entropy suggests the maximum model detection capability. We conduct extensive experiments on ADBench (including 47 real datasets), and the overall results indicate that AutoEncoder (AE) enhanced by our approach not only achieves better performance than ensemble AEs but also requires under 2\% of training time. Lastly, our proposed metric and early-stopping approach are evaluated on other deep OD models, exhibiting their broad potential applicability.

Semi-supervised Learning from Only Positive and Unlabeled Data Using Entropy

Generalized entropy based semi-supervised learning

Positive and Unlabeled Learning through Negative Selection and Imbalance-aware Classification

Semi-Supervised Text Classification Using Positive and Unlabeled Data

Semisupervised learning using negative labels.

A method on selecting reliable samples based on fuzziness in positive and unlabeled learning

Semantic and Generalized Entropy Loss Functions for Semi-Supervised Deep Learning

An Embarrassingly Simple Approach to Semi-Supervised Few-Shot Learning

Learning with Positive and Unlabeled Examples Using Topic-Sensitive PLSA.

Negatives Make a Positive: an Embarrassingly Simple Approach to Semi-Supervised Few-Shot Learning.

Semi-Supervised Semantic Segmentation Using Unreliable Pseudo-Labels

EntropyStop: Unsupervised Deep Outlier Detection with Loss Entropy

InfoMatch: Entropy Neural Estimation for Semi-Supervised Image Classification

Multi-Class Character Classification with Semi-Supervised Learning Based on Information Entropy

Exploiting Unlabeled Data via Partial Label Assignment for Multi-Class Semi-Supervised Learning

A general semi-parametric elliptical distribution model for semi-supervised learning

How Does Unlabeled Data Provably Help Out-of-Distribution Detection?

Semi-supervised learning with extremely sparse labeled data on multiple semi-supervised assumptions

Unsupervised learning of finite mixtures using entropy regularization and its application to image segmentation

Self Adaptive Threshold Pseudo-labeling and Unreliable Sample Contrastive Loss for Semi-supervised Image Classification

Partially Supervised Text Classification with Multi-Level Examples.