Abstract:Data-defect would affect the data quality and the analysis results of data mining. This paper presents a data-defect inspection method with kernel-neighbor-density-change outlier factor (KNDCOF). The definition of kernel neighbor density is proposed to represent the density of each object in database, and the ascending distance series (ADS) of each object is calculated based on the kernel distance between the object and its neighbors. Then, the average density fluctuation (ADF) of the object is established according to the weighted sum of the square of density difference between the object and others in ADS. Finally, the KNDCOF of the object is equal to the ratios of the ADF of the object and the average ADF of neighbors of the object. The degree of the object being an outlier is indicated by the KNDCOF value. The experiments are performed on three real data sets to evaluate the effectiveness of the proposed method. The experimental results verify that the proposed method has higher quality of data-defect inspection and does not increase the time complexity. Note to Practitioners-Data-defect inspection is an important procedure of data preprocessing for a real industrial process. This paper presents a data-defect inspection method with kernel-neighbor-density-change outlier factor to identify the outliers, and addresses the challenges associated with the strong correlation and the nonlinearity of the industrial data. The proposed method calculates the outlier factor for each object, which quantifies how outlying it is. The outlier factor is based on the density difference between the object and its neighbors. The larger the outlier factor of an object is, the higher the outlierness of the object is. The proposed method could be wildly used in an industrial complex data set with different density regions. In the industrial field, engineers can deal with the objects with high outlier factor values based on the actual requirements.

A Method for Measurement Data Modeling and High-Dimensional Outlier Detection Based on Large Dimensional Matrix

An Outlier Detection Method in High Dimensional Time Series Based on LLM

Robust Subspace Outlier Detection in High Dimensional Space

A neighborhood weighted-based method for the detection of outliers

A fast MST-inspired kNN-based outlier detection method

Dimensionality-Aware Outlier Detection: Theoretical and Experimental Analysis

Detecting Outliers in High Dimensional Data Sets using Z-Score Methodology

A New Outlier Detection Model Using Random Walk On Local Information Graph

A Centroid-Based Outlier Detection Method

Comparison of Data Visualization, Outlier Detection and Data Dimensionality Reduction Methods

Improved Method for Noise Detection by DBSCAN and Angle Based Outlier Factor in High Dimensional Datasets

Sample Weighting: an Inherent Approach for Outlier Suppressing Discriminant Analysis

Projected outlier detection in high-dimensional mixed-attributes data set

A Spectral Clustering Based Outlier Detection Technique.

A Fast Outlier Detection Method for Big Data.

Distance-based outlier detection for high dimension, low sample size data

A Parametric and Non-Parametric Approach for High-Accurate Outlier Detection.

Query-Based Outlier Detection in Heterogeneous Information Networks

A Unified Subspace Outlier Ensemble Framework for Outlier Detection in High Dimensional Spaces

Data-Defect Inspection with Kernel-Neighbor-Density-Change Outlier Factor.

Weighted subspace anomaly detection in high-dimensional space