Abstract:Most of existing outlier detection methods assume that the outlier factors (i.e., outlierness scoring measures) of data entities (e.g., feature values and data objects) are Independent and Identically Distributed (IID). This assumption does not hold in real-world applications where the outlierness of different entities is dependent on each other and/or taken from different probability distributions (non-IID). This may lead to the failure of detecting important outliers that are too subtle to be identified without considering the non-IID nature. The issue is even intensified in more challenging contexts, e.g., high-dimensional data with many noisy features. This work introduces a novel outlier detection framework and its two instances to identify outliers in categorical data by capturing non-IID outlier factors. Our approach first defines and incorporates distribution-sensitive outlier factors and their interdependence into a value-value graph-based representation. It then models an outlierness propagation process in the value graph to learn the outlierness of feature values. The learned value outlierness allows for either direct outlier detection or outlying feature selection. The graph representation and mining approach is employed here to well capture the rich non-IID characteristics. Our empirical results on 15 real-world data sets with different levels of data complexities show that (i) the proposed outlier detection methods significantly outperform five state-of-the-art methods at the 95%/99% confidence level, achieving 10%-28% AUC improvement on the 10 most complex data sets; and (ii) the proposed feature selection methods significantly outperform three competing methods in enabling subsequent outlier detection of two different existing detectors.

Detecting Outliers in Sliding Window over Categorical Data Streams

Mining Fuzzy Association Rules in Data Streams

Attribute outlier detection over data streams

Approximate mining of global closed frequent itemsets over data streams

A New Algorithm for Frequency Tendency Prediction over Data Streams

UWFP-Outlier: an efficient frequent-pattern-based outlier detection method for uncertain weighted data streams

The Application of a Double CUSUM Algorithm in Industrial Data Stream Anomaly Detection

FCI-Outlier: An Efficient Frequent Closed Itemset-Based Outlier Detecting Approach on Data Stream

Finding Frequent Closed Itemsets in Sliding Window in Linear Time.

MWFP-outlier: Maximal weighted frequent-pattern-based approach for detecting outliers from uncertain weighted data streams

An Algorithm for Data Stream Speed Anomaly Detection Based on RCSW

An Algorithm for Data Stream Sampling Based on Ring Circular Sliding Window Tightly-Coupled with Buffer

Fuzzy Frequent Pattern Mining Algorithm Based on Weighted Sliding Window and Type-2 Fuzzy Sets over Medical Data Stream

Frequent Items Mining Based on Weight in Data Stream

Continuous Angle-based Outlier Detection on High-dimensional Data Streams.

Homophily Outlier Detection in Non-IID Categorical Data

An efficient approach for outlier detection from uncertain data streams based on maximal frequent patterns

Efficient Discovery of Emerging Frequent Patterns in ArbitraryWindows on Data Streams

Novel structures for counting frequent items in time decayed streams

Robust High-dimensional Bioinformatics Data Streams Mining by ODR-ioVFDT

Finding Frequent Items in Time Decayed Data Streams.