Abstract:Outlier detection is of vital importance in data mining tasks, with numerous applications, including video surveillance and credit card fraud detection. Quite a few outlier detection algorithms have been developed and have received considerable attention, and most existing methods are classified as distance-based algorithms and density-based algorithms. However, both of these approaches have some flaws. The former has difficulty detecting local outliers, and the latter cannot handle low-density pattern problems. Moreover, outlier detection algorithms are sensitive to parameter settings. This paper proposes a simple and efficient outlier detection approach (called ADD) based on the average divergence difference of data objects; in this method there is no need to artificially define the number of neighbors of objects k to solve the above issues. In this algorithm, two new measures, called the divergence factor (DF) and the average divergence difference (LADD), are developed based on the skewed distribution characteristics of data objects and their natural neighbors, thus improving the accuracy of local outlier detection from an innovative research perspective. These factors are presented as external and internal characterization factors because the former characterizes the skew distribution characteristics and compactness relationship of data objects and the latter represents the difference in the skew distribution characteristics of data objects in a neighborhood. Then, we set an appropriate threshold to distinguish whether a data point is an outlier, which eliminates the interference of the Top-N problem. Finally, the final experimental results show that the ADD algorithm achieves an overall improvement in local outlier detection, especially in the detection of outliers in some datasets with complex distributions and in low-density areas, compared to that achieved by state-of-the-art algorithms.

An Improved Linear Algorithm for Discovering Deviation Patterns in Databases

Real-time Algorithm for Trend Analysis of Dynamic Data Streams

The Application of a Double CUSUM Algorithm in Industrial Data Stream Anomaly Detection

Statistical Log Differencing

Robust Iterative Algorithm of Weighted Least Squares Support Vector Machine and Its Application in Spectral Analysis

Anomaly Detection Algorithm Based on Electric Equipment

Efficient Search of Reliable Exceptions.

ADD: a new average divergence difference-based outlier detection method with skewed distribution of data objects

An Improved Piecewise Linear Online Trending Algorithm and Its Application

A Demonstration of DLBD: Database Logic Bug Detection System.

A Pattern of Experiment Data Mining Based on Π Theorem Combined with ANN

Research of Detection Algorithm for Time Series Abnormal Subsequence

Improved incremental local outlier detection for data streams based on the landmark window model

A neighborhood weighted-based method for the detection of outliers

LDBOD: A novel local distribution based outlier detector

An Improved Similarity Comparison Method for Long Time Series

A Real-Time Computer Network Trend Analysis Algorithm Based on Dynamic Data Stream in the Context of Big Data

Management and selection of algorithms based on database

Research of Dynamical Detecting Technique of Non-Functional Dependence Program Invariant

Sequential Change Detection by Optimal Weighted $\ell_2$ Divergence

Finding surprising patterns in a time series database in linear time and space