Abstract:Outlier detection is critically important in the field of data mining. Real-world data have the impreciseness and ambiguity which can be handled by means of rough set theory. Information entropy is an effective way to measure the uncertainty in an information system. Most outlier detection methods may be called unsupervised outlier detection because they are only dealt with unlabeled data. When sufficient labeled data are available, these methods are used in a decision information system, which means that the decision attribute is discarded. Thus, these methods maybe not right for outlier detection in a a decision information system. This paper proposes supervised outlier detection using conditional information entropy and rough set theory. Firstly, conditional information entropy in a decision information system based on rough set theory is calculated, which provides a more comprehensive measure of uncertainty. Then, the relative entropy and relative cardinality are put forward. Next, the degree of outlierness and weight function are presented to find outlier factors. Finally, a conditional information entropy-based outlier detection algorithm is given. The performance of the given algorithm is evaluated and compared with the existing outlier detection algorithms such as LOF, KNN, Forest, SVM, IE, and ECOD. Twelve data sets have been taken from UCI to prove its efficiency and performance. For example, the AUC value of CIE algorithm in the Hayes data set is 0.949, and the AUC values of LOF, KNN, SVM, Forest, IE and ECOD algorithms in the Hayes data set are 0.647, 0.572, 0.680, 0.676, 0.928 and 0.667, respectively. The advantage of the proposed outlier detection method is that it fully utilizes the decision information.

Identification and Processing Methods for Outliers in Information Monitoring Systems

A novel outlier detection method for monitoring data in dam engineering

Outlier detection for incomplete real-valued data via information entropy and class-consistent technology

Outlier detection using conditional information entropy and rough set theory

Comparative Analysis of three outlier detection methods in univariate data sets

Intelligent Identification and Order-Sensitive Correction Method of Outliers from Multi-Data Source Based on Historical Data Mining

Detection of outlying patterns from sparse and irregularly sampled electronic health records data

Outliers Learning And Its Applications

A Kalman Filter Based Approach for Outlier Detection in Sensor Networks

A method for outlier detection based on cluster analysis and visual expert criteria

Analytical method for detecting outlier evaluators

A critical overview of outlier detection methods

Outlier Detection and Spatial Analysis Algorithms

An Optimization Model for Outlier Detection in Categorical Data

Human-in-the-loop Outlier Detection.

Improvement of Identification of Blast Furnace Ironmaking Process by Outlier Detection and Missing Value Imputation

Outlier detection in spatial error models using modified thresholding-based iterative procedure for outlier detection approach

Study and Application of Method of Extracting Outliers in Statistical Monitoring Model Based on Robust Scaling

Outlier Detection as Instance Selection Method for Feature Selection in Time Series Classification

Outlier Detection Method for Time Series Based on the Rate of Signal Change

Robust Monitoring of Industrial Processes Using Process Data with Outliers and Missing Values