Abstract:Big data techniques have been applied to power grid for the prediction and evaluation of grid conditions. However, the raw data quality can rarely meet the requirement of precise data analytics since raw data set usually contains samples with missing data to which the common data mining models are sensitive. Besides, the raw training data from a single monitoring system, e.g. dissolved gas analysis (DGA), are rarely sufficient for training in the form of valid instances since raw data set usually contains samples with noisy data. Though classic methods like neural network can be used to fill the gaps of missing data and classify the fault type, their models often fail to fit the rules of power grid conditions. This paper presents an integrated data preprocessing framework (DPF) based on Apache Spark to improve the prediction accuracy for data sets with missing data points and classification accuracy with noise data as well as to meet the big data requirement, which mainly combines missing data prediction, data fusion, data cleansing and fault type classification. First, the prediction model is trained based on the linear regression (LinR). Afterwards, we propose an optimized linear method (OLR) to improve the prediction accuracy. Then, to better utilize the strong correlation among different data sources, new data features extracted by persons correlation coefficient (PCC) are fused into a training data set. Next, principal component analysis (PCA) is taken to reduce the side effect brought by the new feature as well as retaining significant information for classification. Finally, the classification model based on logistic regression (LogR) and support vector machine (SVM) is trained to classify the fault type of electric equipment. We test the DPF framework on missing data prediction and fault type classification of power transformers in power grid system. The experimental results show that the predictors based on the proposed framework achieve lower mean square error and the classifiers obtain higher accuracy than traditional ones. Besides, the training time required for training large-scale data shows a decreasing trend. Therefore, the data preprocessing framework DPF would be a good candidate to predict the missing data and classify the fault type in power grid system.

Predicting Failures of High Voltage Electric Power Facilities via Multidimensional Information Analysis

Data-Based Line Trip Fault Prediction in Power Systems Using LSTM Networks and SVM.

Nonparametric Regression-Based Failure Rate Model for Electric Power Equipment Using Lifecycle Data

Predicting Failures in Power Grids: The Case of Static Overloads

Power Grid Missing Data Filling Method Based on Historical Data Mining Assisted Multi-dimensional Scenario Analysis

Vulnerability Assessment of Important Infrastructure of Power Grid to Natural Disasters Based on Deep Learning

A novel approach of fault-influencing factors analysis for high-voltage switchgears quality supervision based on industrial big data

Artificial Intelligence-based Digital Fault Diagnosis and Prediction for Power Grids

Analysis of Interlocking Faults and Reliability Evaluation Methods for Communication Networks of Power Information Physical Systems

Novel Approach for Fault Diagnosis of the Power Grid with Information Fusion of Multi-data Resources

A Dynamic Risk-Early-warning Methodology of Distribution System Faults Incorporating Spatiotemporal Imbalanced Data Distributions

Combining Unsupervised and Supervised Learning for Asset Class Failure Prediction in Power Systems

Research on intelligent power grid operation and maintenance and fault prediction system based on artificial intelligence

Wind turbine failure prediction and health assessment based on adaptive maximum mean discrepancy

Multi-Scale Temporal Analysis for Failure Prediction in Energy Systems

Integrating Real-Time Monitoring and Asset Health Prediction for Power Transformer Intelligent Maintenance and Decision Support

Multiple Failures Analysis for Complex Electric Power System

Predictive maintenance system for high-end equipment in nuclear power plant under limited degradation knowledge

Hierarchical Fault Diagnosis Approach for Power Grid with Information Fusion Using Multi-data Resources

Failure analysis of photovoltaic strings by constructing a digital multi-twin integrating theory, features, and vision

An Integrated Data Preprocessing Framework Based on Apache Spark for Fault Diagnosis of Power Grid Equipment