Abstract:In recent years, with the advancement of the national construction of smart grid and the development of power grid enterprise integration system, the traditional power data platform has some defect, such as insufficient scalability, repeated implementation of two sets of logic of off-line data warehouse and real-time data warehouse, highly correlated storage of data warehouse during computation, and difficulty in platform migration, which restrict the deeper mining and analysis of power big data. Therefore, this paper adopts a technical architecture incorporating distributed calculating, micro-service, flow-batch integration and lake-warehouse integration, combining the current popular Web front-end and back-end technologies such as Vue, SpringCloud and Flask, and utilizing big data components such as Hadoop, Flink, Hudi and Kafka as well as Docker containerization technology. We explore and build a power grid system operation oriented big data platform that assembles real-time computing, multi-source heterogeneous data storage, and develop APIs for authority management, information management, data analysis and visualization, machine learning and other data supporting services that assist power enterprises in managing electrical equipment and users' power consumption.With the help of this platform, power enterprises can intelligently track the equipment status, view the offline and real-time visual charts to analyze the power consumption by users, so as to have better basis in the power dispatching, maintenance and scheme adjustment of electrical market.

Practice and Application of Distributed Data Quality Management System in Power Enterprise

Design and Implementation of Clinical Data Integration and Management System Based on Hadoop Platform

Distributed Model Based on Data Partition and Load Balance Algorithm

Data Based Application Partitioning and Workload Balance in Distributed Environment

Architecture Design of Wide-area Distributed Real-time Database in Grid Dispatching Control System

QDFS: A Quality-Aware Distributed File Storage Service Based on HDFS

Data Quality Transaction On Different Distributed Ledger Technologies

The Research of Distributed Shared Memory Technology in Power System

Design and Implementation of Power Big Data Platform

SparkDQ: Efficient Generic Big Data Quality Management on Distributed Data-Parallel Computation

Application of Data Warehouse to Power System

Comprehensive Evaluation of Big Data Quality in Power Systems with Entropy Weight and Grey System Theory

Design Of Distributed Data Warehouse For Sales Decision Of Large-Scale Clothing Enterprise

Power Big Data Analysis Platform Design Based on Hadoop

Practice and Reflection on Data-Driven Modeling in Electric Power Domain

Big Data Analysis Supports the Research and Pilot Application of Distribution Network Planning Technology

Toward Spreadsheet-Based Data Management in Distributed Enterprise Environment

APPLICATION OF DATA WAREHOUSE AND DATA MINING TECHNIQUES TO POWER SYSTEMS

Research on Key Technologies of Data Warehouse Construction in Unified Data Center in Electric Power Enterprises

Data Cleaning for Power Quality Monitoring

Big Data Quality Prediction in the Process Industry: A Distributed Parallel Modeling Framework