Abstract:In the era of big data, practical applications in various domains continually generate large-scale time-series data. Among them, some data show significant or potential periodicity characteristics, such as meteorological and financial data. It is critical to efficiently identify the potential periodic patterns from massive time-series data and provide accurate predictions. In this paper, a Periodicity-based Parallel Time Series Prediction (PPTSP) algorithm for large-scale time-series data is proposed and implemented in the Apache Spark cloud computing environment. To effectively handle the massive historical datasets, a Time Series Data Compression and Abstraction (TSDCA) algorithm is presented, which can reduce the data scale as well as accurately extracting the characteristics. Based on this, we propose a multi-layer time series periodic pattern recognition (MTSPPR) algorithm using the Fourier Spectrum Analysis (FSA) method. In addition, a Periodicity-based Time Series Prediction (PTSP) algorithm is proposed. Data in the subsequent period are predicted based on all previous period models, in which a time attenuation factor is introduced to control the impact of different periods on the prediction results. Moreover, to improve the performance of the proposed algorithms, we propose a parallel solution on the Apache Spark platform, using the Streaming real-time computing module. To efficiently process the large-scale time-series datasets in distributed computing environments, Distributed Streams (DStreams) and Resilient Distributed Datasets (RDDs) are used to store and calculate these datasets. Logical and data dependencies of RDDs in the P-TSDCA, P-MTSPPR, and P-PTSP processes are considered, and the corresponding parallel execution solutions are conducted. Extensive experimental results show that our PPTSP algorithm has significant advantages compared with other algorithms in terms of prediction accuracy and performance.

Predicting breakdowns in cloud services (with SPIKE)

Microservice Auto-Scaling Algorithm Based on Workload Prediction in Cloud-Edge Collaboration Environment

Sage: Leveraging ML to Diagnose Unpredictable Performance in Cloud Microservices

Cloud Computing: A Model Construct of Real-Time Monitoring for Big Dataset Analytics Using Apache Spark

Sage: Using Unsupervised Learning for Scalable Performance Debugging in Microservices

Predicting Scheduling Failures in the Cloud

Predicting Node Failures in an Ultra-Large-Scale Cloud Computing Platform

A Failure Prediction Approach Based on BiLSTM and Deep Feature Extractor for Hard Disk Drives.

Fault Diagnosis for Service Composition by Spiking Neural P Systems with Colored Spikes

Machine Failure Prediction Using Survival Analysis

A periodicity-based parallel time series prediction algorithm in cloud computing environments

Integrating System State into Spatio Temporal Graph Neural Network for Microservice Workload Prediction

A Short-term Prediction Based Elastic Service Scaling Algorithm

Predicting DRAM-Caused Node Unavailability in Hyper-Scale Clouds

TR-Spark

DC-Prophet: Predicting Catastrophic Machine Failures in DataCenters

Probabilistic Failure Detection for Efficient Distributed Storage Maintenance

Machine Learning for Performance Prediction of Spark Cloud Applications

PREVENT: An Unsupervised Approach to Predict Software Failures in Production

Augmented intelligence forecasting and what-if-scenario analytics with quantified uncertainty for big real-time slope monitoring data

Leveraging Deep Learning to Improve Performance Predictability in Cloud Microservices with Seer.