Abstract:Trend forecasting is vital to optimize the workload performance. It becomes even more urgent with an increasing number of applications and database configurations. However, DBAs mainly target at historical workloads and may give suboptimal configuration advice when the workload trends have changed. Although there are some studies on trend forecasting, they have several limitations. First, they mainly predict the changes of query numbers, which do not combine other critical factors (e.g., disk utilization) and cannot fully reflect the future workload trends. Besides, there are numerous queries in the workloads and exact clustering algorithms like K-means cannot effectively merge similar queries which contain noises like time shifts. Second, basic machine learning models like RNN may have relatively low prediction accuracy on complex workloads (e.g., no cycles but random bursts). Third, real-world workloads may have diverse patterns, while previous models cannot efficiently and reliably predict for all the different workload patterns.To address these challenges, we propose a trend forecasting system (DBAugur) that utilizes adversarial neural networks to predict the trends of different workloads. First, DBAugur collects the important features (e.g., queries, resource metrics) to characterize workloads, and reduces the number of involved queries by separately merging similar queries based on the SQL semantics and trend patterns. Second, DBAugur utilizes Generative Adversarial Networks (GANs) to capture the latent patterns, correlations between different metrics, and occasional bursts within the complicated and time-varying workloads. Moreover, we further propose a time-sensitive ensemble algorithm that takes advantage of various machine learning models (e.g., generative models, convolutional models, feed-forward models) to accommodate the various workload patterns. The experimental results show that DBAugur outperformed state-of-the-art methods on various real-world workloads.

LightPro: Lightweight Probabilistic Workload Prediction Framework for Database-as-a-Service

Robust Auto-Scaling with Probabilistic Workload Forecasting for Cloud Databases

Load Prediction for Data Centers Based on Database Service.

A Lightweight General Adaptive Optimization Tool for Relational DBMSs under HTAP Workloads

Query-based Workload Forecasting for Self-Driving Database Management Systems

Workload Forecasting Framework for Applications in Cloud

Adaptive Recollected RNN for Workload Forecasting in Database-as-a-Service

An Adaptive Prediction Approach Based on Workload Pattern Discrimination in the Cloud.

Towards Workload Trend Time Series Probabilistic Prediction Via Probabilistic Deep Learning.

A Self-Adaptive Prediction Algorithm for Cloud Workloads

CloudProphet: A Machine Learning-Based Performance Prediction for Public Clouds

TEALED: A Multi-Step Workload Forecasting Approach Using Time-Sensitive EMD and Auto LSTM Encoder-Decoder

Resource and Performance Prediction at High Utilization for N-Tier Cloud-Based Service Systems.

Real-time Workload Pattern Analysis for Large-scale Cloud Databases

Forecasting Workload in Cloud Computing: Towards Uncertainty-Aware Predictions and Transfer Learning

A Workload Prediction-Based Multi-Vm Provisioning Mechanism in Cloud Computing

Performance and Resource Prediction at High Utilization for N-Tier Service Systems in Cloud an Experiment Driven Approach.

DProbe: Profiling and Predicting Multi-tenant Deep Learning Workloads for GPU Resource Scaling.

Auto-Scaling Provision Basing on Workload Prediction in the Virtualized Data Center

DBAugur: an Adversarial-based Trend Forecasting System for Diversified Workloads.

WorkloadDiff: Conditional Denoising Diffusion Probabilistic Models for Cloud Workload Prediction