Abstract:Non-Intrusive Load Monitoring (NILM) comprises of a set of techniques that provide insights into the energy consumption of households and industrial facilities. Latest contributions show significant improvements in terms of accuracy and generalisation abilities. Despite all progress made concerning disaggregation techniques, performance evaluation and comparability remains an open research question. The lack of standardisation and consensus on evaluation procedures makes reproducibility and comparability extremely difficult. In this paper, we draw attention to comparability in NILM with a focus on highlighting the considerable differences amongst common energy datasets used to test the performance of algorithms. We divide discussion on comparability into data aspects, performance metrics, and give a close view on evaluation processes. Detailed information on pre-processing as well as data cleaning methods, the importance of unified performance reporting, and the need for complexity measures in load disaggregation are found to be the most urgent issues in NILM-related research. In addition, our evaluation suggests that datasets should be chosen carefully. We conclude by formulating suggestions for future work to enhance comparability.

What problem does this paper attempt to address?

This paper attempts to solve the comparability problem in non - invasive load monitoring (NILM) research. Specifically, although significant progress has been made in load disaggregation techniques, there are still open research problems in performance evaluation and result comparability. The paper points out that the lack of standardized and consensual evaluation procedures makes it extremely difficult to reproduce results and compare different studies. To improve this situation, the paper focuses on the significant differences between common energy data sets and discusses the comparability problem from three aspects: data set characteristics, performance indicators, and evaluation processes. The main contributions of the paper include: 1. **Data set analysis**: It compares in detail the characteristics of several commonly used low - sampling - rate energy data sets, such as duration, sampling rate, and the number of installed sub - meters, and points out the impact of these characteristics on the performance of load disaggregation algorithms. 2. **Performance indicators**: It emphasizes the importance of choosing appropriate performance indicators and suggests using normalized indicators to compare electrical appliances of different power levels. 3. **Evaluation process**: It proposes two new evaluation indicators - test set ratio (TSR) and event ratio (EVR) - to quantify the amount of data and the number of events used in the evaluation process, thereby improving the comparability of evaluation results. 4. **Noise level**: It suggests evaluating the noise level in the aggregated power signal and introduces the noise - to - aggregation ratio (NAR) as a measurement standard. Through these methods, the paper aims to provide suggestions for improving comparability in future NILM research, especially when using closed - source data sets or when privacy issues are involved.

Towards Comparability in Non-Intrusive Load Monitoring: On Data and Performance Evaluation

A Survey of the Research on Non-intrusive Load Monitoring and Disaggregation

On Metrics to Assess the Transferability of Machine Learning Models in Non-Intrusive Load Monitoring

NILMTK: An Open Source Toolkit for Non-intrusive Load Monitoring

Non-intrusive Load Monitoring and Its Challenges in a NILM System Framework

Non-Intrusive Load Monitoring System Framework and Load Disaggregation Algorithms: A Survey

A Field Study of Nonintrusive Load Monitoring Devices and Implications for Load Disaggregation

Towards Trustworthy Energy Disaggregation: A Review of Challenges, Methods, and Perspectives for Non-Intrusive Load Monitoring

Non-Intrusive Load Monitoring in Smart Grids: A Comprehensive Review

A Comprehensive Review on the NILM Algorithms for Energy Disaggregation

Complexity of Power Draws for Load Disaggregation

Non-intrusive Load Monitoring Based on Equipment Operation State

A comparison of non-intrusive load monitoring methods for commercial and residential buildings

A review of current methods and challenges of advanced deep learning-based non-intrusive load monitoring (NILM) in residential context

Non-intrusive Load Monitoring for Household Energy Disaggregation: A State-of-the-Art

An End-Cloud Collaborated Framework for Transferable Non-Intrusive Load Monitoring

Low-Complexity Non-Intrusive Load Monitoring Using Unsupervised Learning and Generalized Appliance Models

DiffNILM: A Novel Framework for Non-Intrusive Load Monitoring Based on the Conditional Diffusion Model

How Can Non-Intrusive Load Monitoring Contribute to the Assessment of the Smart Readiness Indicator?

Multi-State Energy Classifier to Evaluate the Performance of the NILM Algorithm

Resilient data-driven non-intrusive load monitoring for efficient energy management using machine learning techniques