Abstract:The rapid development in data science and the increasing availability of building operational data have provided great opportunities for developing data-driven solutions for intelligent building energy management. Data preprocessing serves as the foundation for valid data analyses. It is an indispensable step in building operational data analysis considering the intrinsic complexity of building operations and deficiencies in data quality. Data preprocessing refers to a set of techniques for enhancing the quality of the raw data, such as outlier removal and missing value imputation. This article serves as a comprehensive review of data preprocessing techniques for analysing massive building operational data. A wide variety of data preprocessing techniques are summarised in terms of their applications in missing value imputation, outlier detection, data reduction, data scaling, data transformation, and data partitioning. In addition, three state-of-the-art data science techniques are proposed to tackle practical data challenges in the building field, i.e., data augmentation, transfer learning, and semi-supervised learning. In-depth discussions have been presented to describe the pros and cons of existing preprocessing methods, possible directions for future research and potential applications in smart building energy management. The research outcomes are helpful for the development of data-driven research in the building field.

The Theory and Methods of Data Preparation: an Overview

Research on Data Preprocessing Methods for Big Data

Data Preprocessing: The Techniques for Preparing Clean and Quality Data for Data Analytics Process

Study on Data Preprocessing Arithmetic and Its Applications in Enterprise

Pre-Processing: A Data Preparation Step

Research on the Methods of Data Reduction in Data Preprocessing

Data Cleaning In Data Warehouse: A Survey of Data Pre-processing Techniques and Tools

Research of Data Preprocessing in Education Decision Support System

Automated Data Cleaning For Data Centers: A Case Study

Data analysis and preprocessing techniques for air quality prediction: a survey

Univariate Description and Bivariate Statistical Inference: the First Step Delving into Data

DATA PRETREATMENT IN INTELLIGENT MODELING METHODS

The potential and perils of preprocessing: Building new foundations

Data_prepare.csv

The Data Preparation Process in Real Estate: Guidance and Review

Data Preprocessing for Building Hospital Data Warehouse

Influence of Data Preprocessing.

DATA PRE-PROCESSING TECHNIQUES IN DATA MINING: A REVIEW

A Method to Screen, Assess, and Prepare Open Data for Use

A Review on Data Preprocessing Techniques Toward Efficient and Reliable Knowledge Discovery From Building Operational Data

A Primer on the Data Cleaning Pipeline