Abstract:In the era of big data, practical applications in various domains continually generate large-scale time-series data. Among them, some data show significant or potential periodicity characteristics, such as meteorological and financial data. It is critical to efficiently identify the potential periodic patterns from massive time-series data and provide accurate predictions. In this paper, a Periodicity-based Parallel Time Series Prediction (PPTSP) algorithm for large-scale time-series data is proposed and implemented in the Apache Spark cloud computing environment. To effectively handle the massive historical datasets, a Time Series Data Compression and Abstraction (TSDCA) algorithm is presented, which can reduce the data scale as well as accurately extracting the characteristics. Based on this, we propose a multi-layer time series periodic pattern recognition (MTSPPR) algorithm using the Fourier Spectrum Analysis (FSA) method. In addition, a Periodicity-based Time Series Prediction (PTSP) algorithm is proposed. Data in the subsequent period are predicted based on all previous period models, in which a time attenuation factor is introduced to control the impact of different periods on the prediction results. Moreover, to improve the performance of the proposed algorithms, we propose a parallel solution on the Apache Spark platform, using the Streaming real-time computing module. To efficiently process the large-scale time-series datasets in distributed computing environments, Distributed Streams (DStreams) and Resilient Distributed Datasets (RDDs) are used to store and calculate these datasets. Logical and data dependencies of RDDs in the P-TSDCA, P-MTSPPR, and P-PTSP processes are considered, and the corresponding parallel execution solutions are conducted. Extensive experimental results show that our PPTSP algorithm has significant advantages compared with other algorithms in terms of prediction accuracy and performance.

Effective Prediction of Missing Data on Apache Spark over Multivariable Time Series.

Temporal Dynamic Matrix Factorization for Missing Data Prediction in Large Scale Coevolving Time Series

Incomplete Electrocardiogram Time Series Prediction

Forecasting in multivariate irregularly sampled time series with missing values

Power Grid Missing Data Filling Method Based on Historical Data Mining Assisted Multi-dimensional Scenario Analysis

Time-delayed Multivariate Time Series Predictions.

Bayesian Temporal Factorization for Multidimensional Time Series Prediction

DBT-DMAE: An Effective Multivariate Time Series Pre-Train Model under Missing Data

Deep Learning for Multivariate Time Series Imputation: A Survey

Diffusion Models for Multivariate Time Series Generation with Missing Values

On-Line Imputation For Missing Values

TriD-MAE: A Generic Pre-trained Model for Multivariate Time Series with Missing Values

LIFE: Learning Individual Features for Multivariate Time Series Prediction with Missing Values

Any-Time Methods for Time-Series Prediction with Missing Observations

Autoregressive-Model-Based Methods for Online Time Series Prediction with Missing Values: an Experimental Evaluation

Big Data Driven Marine Environment Information Forecasting: A Time Series Prediction Network

Missing Value Imputation on Multidimensional Time Series

Incomplete Time Series Prediction Using Max-Margin Classification of Data with Absent Features

An End-to-End Model for Time Series Classification In the Presence of Missing Values

A periodicity-based parallel time series prediction algorithm in cloud computing environments

Improving Power Grid Monitoring Data Quality: an Efficient Machine Learning Framework for Missing Data Prediction