Abstract:Outlier detection is of vital importance in data mining tasks, with numerous applications, including video surveillance and credit card fraud detection. Quite a few outlier detection algorithms have been developed and have received considerable attention, and most existing methods are classified as distance-based algorithms and density-based algorithms. However, both of these approaches have some flaws. The former has difficulty detecting local outliers, and the latter cannot handle low-density pattern problems. Moreover, outlier detection algorithms are sensitive to parameter settings. This paper proposes a simple and efficient outlier detection approach (called ADD) based on the average divergence difference of data objects; in this method there is no need to artificially define the number of neighbors of objects k to solve the above issues. In this algorithm, two new measures, called the divergence factor (DF) and the average divergence difference (LADD), are developed based on the skewed distribution characteristics of data objects and their natural neighbors, thus improving the accuracy of local outlier detection from an innovative research perspective. These factors are presented as external and internal characterization factors because the former characterizes the skew distribution characteristics and compactness relationship of data objects and the latter represents the difference in the skew distribution characteristics of data objects in a neighborhood. Then, we set an appropriate threshold to distinguish whether a data point is an outlier, which eliminates the interference of the Top-N problem. Finally, the final experimental results show that the ADD algorithm achieves an overall improvement in local outlier detection, especially in the detection of outliers in some datasets with complex distributions and in low-density areas, compared to that achieved by state-of-the-art algorithms.

Statistical Detection Of Collective Data Fraud

Detecting abnormal situations using the Kullback-Leibler divergence.

Distinguishing Good from Bad: Distributed-Collaborative-Representation-Based Data Fraud Detection in Federated Learning.

Unsupervised Learning Based Distributed Detection of Global Anomalies.

Exploring Distributional Discrepancy for Multidimensional Point Set Retrieval

Research on Dynamic Data Flow Anomaly Detection based on Machine Learning

Subset Multivariate Collective And Point Anomaly Detection

ADD: a new average divergence difference-based outlier detection method with skewed distribution of data objects

Modeling Heterogeneous Statistical Patterns in High-dimensional Data by Adversarial Distributions: an Unsupervised Generative Framework

Dynamic Micro-cluster-Based Streaming Data Clustering Method for Anomaly Detection.

Detecting Regions of Maximal Divergence for Spatio-Temporal Anomaly Detection

Diversity-Measurable Anomaly Detection

An Integrated Cluster Detection, Optimization, and Interpretation Approach for Financial Data

Continuous Adaptive Outlier Detection on Distributed Data Streams.

Data Fusion Approach for Collaborative Anomaly Intrusion Detection in Blockchain-based Systems

BadLink: Combining Graph and Information-Theoretical Features for Online Fraud Group Detection

A Differential Privacy Noise Dynamic Allocation Algorithm for Big Multimedia Data

Distribution-Free Statistical Dispersion Control for Societal Applications

Dynamic Suspicious Financial Transactions Recognition Based on Multi-dimension Analysis of Data Stream

FraudVis: Understanding Unsupervised Fraud Detection Algorithms

Efficient Trustworthiness Management for Malicious User Detection in Big Data Collection