Abstract:Most of existing outlier detection methods assume that the outlier factors (i.e., outlierness scoring measures) of data entities (e.g., feature values and data objects) are Independent and Identically Distributed (IID). This assumption does not hold in real-world applications where the outlierness of different entities is dependent on each other and/or taken from different probability distributions (non-IID). This may lead to the failure of detecting important outliers that are too subtle to be identified without considering the non-IID nature. The issue is even intensified in more challenging contexts, e.g., high-dimensional data with many noisy features. This work introduces a novel outlier detection framework and its two instances to identify outliers in categorical data by capturing non-IID outlier factors. Our approach first defines and incorporates distribution-sensitive outlier factors and their interdependence into a value-value graph-based representation. It then models an outlierness propagation process in the value graph to learn the outlierness of feature values. The learned value outlierness allows for either direct outlier detection or outlying feature selection. The graph representation and mining approach is employed here to well capture the rich non-IID characteristics. Our empirical results on 15 real-world data sets with different levels of data complexities show that (i) the proposed outlier detection methods significantly outperform five state-of-the-art methods at the 95%/99% confidence level, achieving 10%-28% AUC improvement on the 10 most complex data sets; and (ii) the proposed feature selection methods significantly outperform three competing methods in enabling subsequent outlier detection of two different existing detectors.

Multivariate outlier detection based on self-organizing map and adaptive nonlinear map and its application

Outlier Detection of High Dimensional Chemical Engineering Process Data Based on Self-Organizing Map

Multivariate Spatial Outlier Detection

Anomaly Detection Via Self-Organizing Map

Projected outlier detection in high-dimensional mixed-attributes data set

Outlier detection based on improved SOM and its application in power system

Self-organizing Map As a New Method for Clustering and Data Analysis

A Novel Outlier Detection Method for Multivariate Data

Intelligent Identification and Order-Sensitive Correction Method of Outliers from Multi-Data Source Based on Historical Data Mining

Self organising maps for visualising and modelling

A Self-Representation Method with Local Similarity Preserving for Fast Multi-View Outlier Detection

Visual Monitoring of Industrial Operation States Based on Kernel Fisher Vector and Self-organizing Map Networks

An outlier map for Support Vector Machine classification

Self-organizing subspace clustering for high-dimensional and multi-view data

Anomaly Detection Based on Confidence Intervals Using SOM with an Application to Health Monitoring

A fast MST-inspired kNN-based outlier detection method

A neighborhood weighted-based method for the detection of outliers

Provable Self-Representation Based Outlier Detection in a Union of Subspaces

A method for outlier detection based on cluster analysis and visual expert criteria

Homophily Outlier Detection in Non-IID Categorical Data

Dynamic and Adaptive Self Organizing Maps Applied to High Dimensional Large Scale Text Clustering