Abstract:In a large-scale cloud environment, many key performance indicators (KPIs) of entities are monitored in real time. These multivariate time series consist of high-dimensional, high-noise, random and time-dependent data. As a common method implemented in artificial intelligence for IT operations (AIOps), time series anomaly detection has been widely studied and applied. However, the existing detection methods cannot fully consider the influence of multiple factors and cannot quickly and accurately detect anomalies in multivariate KPIs of entities. Concurrently, fine-grained root cause locations cannot be determined for detected anomalies and often require abundant normal data that are difficult to obtain for model training. To solve these problems, we propose a long short-term memory (LSTM)-based semisupervised variational autoencoder (VAE) anomaly detection strategy called LR-SemiVAE. First, LR-SemiVAE uses VAE to perform feature dimension reduction and reconstruction of multivariate time series data and judges whether the entity is abnormal by calculating the reconstruction probability score. Second, by introducing an LSTM network into the VAE encoder and decoder, the model can fully learn the time dependence of multivariate time series. Then, LR-SemiVAE predicts the data labels by introducing a classifier to reduce the dependence on the original labeled data during model training. Finally, by proposing a new evidence lower bound (ELBO) loss function calculation method, LR-SemiVAE pays attention to the normal pattern and ignores the abnormal pattern during training to reduce the time cost of removing random anomaly and noise data. However, due to the limitations of LSTM in learning the long-term dependence of time series data, based on LR-SemiVAE, we propose a transformer-based semisupervised VAE anomaly detection and location strategy called RT-SemiVAE for cluster systems with complex service dependencies. This method learns the long-term dependence of multivariate time series by introducing a parallel multihead attention mechanism transformer, while LSTM is used to capture short-term dependence, and the introduction of parallel computing also markedly reduces model training time. After RT-SemiVAE detects entity anomalies, it traces the root entities according to the obtained service dependence graph and locates the root causes at the indicator level. We verify the strategies by using public data sets and constructing a system prototype. Experimental results show that compared with existing baseline methods, the LR-SemiVAE and RT-SemiVAE strategies can detect anomalies more quickly and accurately and perform fine-grained and accurate localization of the root causes of anomalies.

Robust and Unsupervised KPI Anomaly Detection Based on Highly Sensitive Conditional Variational Auto-Encoders.

Robust and Unsupervised KPI Anomaly Detection Based on Conditional Variational Autoencoder

Unsupervised Anomaly Detection for Intricate KPIs via Adversarial Training of VAE

Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications.

SDFVAE: Static and Dynamic Factorized VAE for Anomaly Detection of Multivariate CDN KPIs

Unsupervised Anomaly Detection Using Variational Auto-Encoder based Feature Extraction

Self-Supervised Variational Graph Autoencoder for System-Level Anomaly Detection

Multi-Channel Multi-Scale Convolution Attention Variational Autoencoder (MCA-VAE): An Interpretable Anomaly Detection Algorithm Based on Variational Autoencoder

Efficient KPI Anomaly Detection Through Transfer Learning for Large-Scale Web Services

Semisupervised anomaly detection of multivariate time series based on a variational autoencoder

Multivariate time series anomaly detection with variational autoencoder and spatial–temporal graph network

VESC: a new variational autoencoder based model for anomaly detection

Rapid Deployment of Anomaly Detection Models for Large Number of Emerging KPI Streams.

LARA: A Light and Anti-overfitting Retraining Approach for Unsupervised Time Series Anomaly Detection

A Hybrid Algorithm Incorporating Vector Quantization and One-Class Support Vector Machine for Industrial Anomaly Detection

Multi-mode non-Gaussian variational autoencoder network with missing sources for anomaly detection of complex electromechanical equipment

A Multimodal Anomaly Detector for Robot-Assisted Feeding Using an LSTM-Based Variational Autoencoder

Statistical Test for Anomaly Detections by Variational Auto-Encoders

Unsupervised Anomaly Detection with Local-Sensitive VQVAE and Global-Sensitive Transformers

NVAE-GAN Based Approach for Unsupervised Time Series Anomaly Detection

VAE-based Deep SVDD for anomaly detection