Abstract:The majority of modern consumer-level energy is generated by real-time smart metering systems. These frequently contain anomalies, which prevent reliable estimates of the series' evolution. This work introduces a hybrid modeling approach combining statistics and a Convolutional Autoencoder with a dynamic threshold. The threshold is determined based on Mahalanobis distance and moving averages. It has been tested using real-life energy consumption data collected from smart metering systems. The solution includes a real-time, meter-level anomaly detection system that connects to an advanced monitoring system. This makes a substantial contribution by detecting unusual data movements and delivering an early warning. Early detection and subsequent troubleshooting can financially benefit organizations and consumers and prevent disasters from occurring.
What problem does this paper attempt to address?
The main objective of this paper is to propose a real-time anomaly detection method for power consumption data in smart meter systems. Specifically, the authors conducted research on the following aspects:
1. **Research Background and Motivation**: With the growth of global energy demand and the increasing contribution of buildings (residential and commercial) to energy consumption, effective energy management has become crucial. Smart meters can record power consumption in real-time, but these data often contain anomalies. These anomalies may be caused by technical, environmental, human, or operational factors, affecting the reliability of the data and the effectiveness of subsequent analysis.
2. **Problem Definition**: The paper aims to develop a hybrid modeling method that combines statistical methods with Convolutional Autoencoders and uses dynamic thresholds to achieve real-time anomaly detection in smart meter data. The goal of this method is to improve detection accuracy, response speed, and adaptability to handle anomaly patterns in real-world scenarios.
3. **Main Contributions**: The paper proposes a method that combines advanced data mining techniques and machine learning (specifically unsupervised Convolutional Autoencoders) to improve system performance through dynamic thresholds (based on Mahalanobis distance and moving average). This method was tested on real-world datasets, demonstrating its effectiveness in real-time anomaly detection.
4. **Methodology**: The authors used a simple Convolutional Autoencoder network architecture for training, which includes three convolutional layers, utilizing techniques such as batch normalization, ReLU activation function, and max pooling. Additionally, they introduced a novel approach that combines reconstruction error and Mahalanobis distance to identify anomaly patterns and used moving average to smooth data noise, establish baselines, and calculate dynamic thresholds.
5. **Results**: Experimental results show that the method successfully detected 622 anomaly points from unseen test datasets, indicating that the proposed framework can be effectively applied to real-time energy monitoring systems, helping to reduce energy waste, prevent economic losses, and avert potential disasters.
In summary, this paper aims to address the anomaly detection problem in smart meter data by proposing a novel and practical hybrid modeling method, providing valuable insights for power consumption monitoring and management.