Enhancing Rainfall Data Consistency and Completeness: A Spatiotemporal Quality Control Approach and Missing Data Reconstruction Using MICE on Large Precipitation Datasets

Golkhatmi, Nafiseh Seyyed Nezhad

DOI: https://doi.org/10.1007/s11269-023-03567-0

IF: 4.426

2024-01-23

Water Resources Management

Abstract:Accurate and complete data are crucial for climate, environmental, water, and agricultural research. Any record of data that is contaminated with errors should be considered missing and reconstructed. Pollution in climate data can lead to systematic errors, such as polluted outlier data. Simply removing outlier data is not a reliable method, and it is important to perform quality control checks to determine the reliability of the data. While methods for detecting outlier data have received significant attention from researchers, less investigation has been conducted on determining the pollution of outlier data. We propose methods for quality control and reconstruction of incomplete rainfall data using data from 141 stations in the Qaraqhum basin in northeastern Iran. We performed checks for gross errors, temporal consistency, and outlier data. As we observed that the probability distribution of monthly precipitation had a skewness shape, we utilized a robust 3σ-rule to detect outlier values. We propose the use of information such as the number of daily precipitation events per month, maximum monthly rainfall, and standardized monthly rainfall (based on robust 3σ-rule) to detect pollution of outlier values. Additionally, we performed a spatial–temporal comparison to determine the difference between no record and no occurrence of precipitation. For data reconstruction, we used the "mice" package in R, which imputes data using chain equations. We investigated the performance of five functions available in the mice package, and the results showed that the "norm.nob" method had the best performance, while the "sample" and "mean" methods had the weakest performance.

water resources,engineering, civil