Abstract:Several techniques for multivariate time series anomaly detection have been proposed recently, but a systematic comparison on a common set of datasets and metrics is lacking. This paper presents a systematic and comprehensive evaluation of unsupervised and semi-supervised deep-learning based methods for anomaly detection and diagnosis on multivariate time series data from cyberphysical systems. Unlike previous works, we vary the model and post-processing of model errors, i.e. the scoring functions independently of each other, through a grid of 10 models and 4 scoring functions, comparing these variants to state of the art methods. In time-series anomaly detection, detecting anomalous events is more important than detecting individual anomalous time-points. Through experiments, we find that the existing evaluation metrics either do not take events into account, or cannot distinguish between a good detector and trivial detectors, such as a random or an all-positive detector. We propose a new metric to overcome these drawbacks, namely, the composite F-score ($Fc_1$), for evaluating time-series anomaly detection. Our study highlights that dynamic scoring functions work much better than static ones for multivariate time series anomaly detection, and the choice of scoring functions often matters more than the choice of the underlying model. We also find that a simple, channel-wise model - the Univariate Fully-Connected Auto-Encoder, with the dynamic Gaussian scoring function emerges as a winning candidate for both anomaly detection and diagnosis, beating state of the art algorithms.

HybridAD: A Hybrid Model-Driven Anomaly Detection Approach for Multivariate Time Series

Hybrid Anomaly Detection via Multihead Dynamic Graph Attention Networks for Multivariate Time Series

CANet: Convolutional Attention-based Network for Multivariate Time Series Anomaly Detection

A Multi-scale Parallel Unsupervised Model for Multivariate Time Series Anomaly Detection

A space-embedding strategy for anomaly detection in multivariate time series

Segmentation-Based Adversarial Denoising Auto-Encoder for Anomaly Detection in Multivariate Time Series Data

DTAAD: Dual Tcn-Attention Networks for Anomaly Detection in Multivariate Time Series Data

An Evaluation of Anomaly Detection and Diagnosis in Multivariate Time Series

Multivariate Time-Series Anomaly Detection via Graph Attention Network

MTS Anomaly Detection Based on Temporal and Feature Correlation

Hierarchical Context Representation and Self-adaptive Thresholding for Multivariate Anomaly Detection

An Empirical Analysis of Anomaly Detection Methods for Multivariate Time Series

Algorithmic Recourse for Anomaly Detection in Multivariate Time Series

Anomaly Detection for Time Series Using VAE-LSTM Hybrid Model

DCFF-MTAD: A Multivariate Time-Series Anomaly Detection Model Based on Dual-Channel Feature Fusion

An enhanced abnormal information expression spatiotemporal model for anomaly detection in multivariate time-series

GenAD: General Representations of Multivariate Time Seriesfor Anomaly Detection

Multivariate Time Series Anomaly Detection with Few Positive Samples

Multivariate Time Series Anomaly Detection and Interpretation Using Hierarchical Inter-Metric and Temporal Embedding

Detection of Anomalies in Multivariate Time Series Using Ensemble Techniques