Abstract:Over the past few years, the scale of sensor networks has greatly expanded. This generates extended spatiotemporal datasets, which form a crucial information resource in numerous fields, ranging from sports and healthcare to environmental science and surveillance. Unfortunately, these datasets often contain missing values due to systematic or inadvertent sensor misoperation. This incompleteness hampers the subsequent data analysis, yet addressing these missing observations forms a challenging problem. This is especially the case when both the temporal correlation of timestamps within a single sensor and the spatial correlation between sensors are important. Here, we apply and evaluate 12 imputation methods to complete the missing values in a dataset originating from large-scale environmental monitoring. As part of a large citizen science project, IoT-based microclimate sensors were deployed for six months in 4400 gardens across the region of Flanders, generating 15-min recordings of temperature and soil moisture. Methods based on spatial recovery as well as time-based imputation were evaluated, including Spline Interpolation, MissForest, MICE, MCMC, M-RNN, BRITS, and others. The performance of these imputation methods was evaluated for different proportions of missing data (ranging from 10% to 50%), as well as a realistic missing value scenario. Techniques leveraging the spatial features of the data tend to outperform the time-based methods, with matrix completion techniques providing the best performance. Our results therefore provide a tool to maximize the benefit from costly, large-scale environmental monitoring efforts.

Missing Data Imputation on IoT Sensor Networks: Implications for On-Site Sensor Calibration

Missing Value Imputation of Wireless Sensor Data for Environmental Monitoring

A New Imputation Technique Based a Multi-Spike Neural Network to Handle Missing Data in the Internet of Things Network (IoT)

Dynamic Measurement and Data Calibration for Aerial Mobile IoT

Deep Compressed Sensing Based Data Imputation for Urban Environmental Monitoring

Missing Data Imputation With Bayesian Maximum Entropy for Internet of Things Applications

Application of Machine Learning Techniques for the Calibration of Low-cost IoT Sensors in Environmental Monitoring Networks.

Missing Value Imputation Based on Gaussian Mixture Model for the Internet of Things

Recover Missing Sensor Data with Iterative Imputing Network

Best Fit Missing Value Imputation (BFMVI) Algorithm for Incomplete Data in the Internet of Things.

Temporal and Spatial Nearest Neighbor Values Based Missing Data Imputation in Wireless Sensor Networks

Data Imputation in IoT Using Spatio-Temporal Variational Auto-Encoder.

Missing data imputation and sensor self-validation towards a sustainable operation of wastewater treatment plants via deep variational residual autoencoders

A Lightweight Window Portion-Based Multiple Imputation for Extreme Missing Gaps in IoT Systems

Reconstruction of Missing Big Sensor Data

On the Performance of Imputation Techniques for Missing Values on Healthcare Datasets

A Graph-Based Approach for Missing Sensor Data Imputation

Multistage Large Segment Imputation Framework Based on Deep Learning and Statistic Metrics

Comparison of Estimating Missing Values in IoT Time Series Data Using Different Interpolation Algorithms

An Efficient Architecture for Imputing Distributed Data Sets of IoT Networks

Improving IoT Data Availability Via Feedback- and Voting-Based Anomaly Imputation