Abstract:In recent years, outliers caused by manual operation errors and equipment acquisition failures often occur, bringing challenges to big data analysis. In view of the difficulties in identifying and correcting outliers of multi-source data, an intelligent identification and order-sensitive correction method of outliers from multi-data sources based on historical data mining was proposed. First, an intelligent identification method of outliers of single-source data is proposed based on neural tangent kernel K-means (NTKKM) clustering. The original data is mapped to high-dimensional feature space using Neural Tangent Kernel, where the features of outliers are acquired by K-means clustering to realize the accurate identification of outliers. Second, an order-sensitive missing value imputation framework for multi-source data (OMSMVI) was proposed. The similarity graph of sources with missing data was constructed based on multidimensional similarity analysis, and the filling order decision was transformed into an optimization problem to realize the optimal filling order decision of missing values in multi-source data. Finally, a neighborhood-based imputation (NI) algorithm is proposed. Based on the traditional KNN filling algorithm, neighboring nodes of sources with missing data are flexibly selected to the achieve accurate correction of outliers. The case experiment was operated on actual power grid data, and the results show that the proposed clustering method can identify outliers more accurately, and the determined optimal imputation sequence has higher accuracy, which provide a feasible new idea for the identification and correction of outliers in the process of data preprocessing.

Outlier detection based on improved SOM and its application in power system

A Method of Data Mining Based on SOM Clustering and Its Application

Outlier Detection Method Based on Compressed Time Series and Voronoi Diagram for Power Data

Research on Clustering Algorithm of Load Decomposition Considering Harmonic Characteristics in Power Safety Monitoring

Detection and Classification of Anomalies in Power Distribution System Using Outlier Filtered Weighted Least Square

Research and application of safety monitoring technology of distribution automation based on SOM neural network

Big data clustering algorithm of power system user load characteristics based on K-means and SOM neural network

Outlier-Detection-Based EKF for Power Systems

Decentralized dynamic state estimation for multi-machine power systems with non-Gaussian noises: Outlier detection and localization

Machine Learning for Complex Systems with Abnormal Pattern by Exception Maximization Outlier Detection Method

Smart Grid Outlier Detection Based on the Minorization-Maximization Algorithm

Data-Driven Event Detection of Power Systems Based on Unequal-Interval Reduction of PMU Data and Local Outlier Factor.

Intelligent Identification and Order-Sensitive Correction Method of Outliers from Multi-Data Source Based on Historical Data Mining

A neighborhood weighted-based method for the detection of outliers

Ordinal Outlier Detection Based On Recursive Uniform Partitioning

Detecting outliers by clustering algorithms

SUOD: Toward Scalable Unsupervised Outlier Detection

Clustering With Outlier Removal

Abnormal detection and early warning mechanism of power market transaction data: a practice and exploration

Parallel and Distributed Spatial Outlier Mining in Grid: Algorithm, Design and Application

A Dynamic and Real-time Outlier Detection Method forEnergy Consumption of Campus Building