Abstract:Cyber–physical systems (CPSs) serve as the pivotal core of Internet of Things (IoT) infrastructures, such as smart grids and intelligent transportation, deploying interconnected sensing devices to monitor operating status. With increasing decentralization, the surge in sensor devices expands the potential vulnerability to cyber attacks. It is imperative to conduct anomaly detection research on the multivariate time series data that these sensors produce to bolster the security of distributed CPSs. However, the high dimensionality, absence of anomaly labels in real-world datasets, and intricate non-linear relationships among sensors present considerable challenges in formulating effective anomaly detection algorithms. Recent deep-learning methods have achieved progress in the field of anomaly detection. Yet, many methods either rely on statistical models that struggle to capture non-linear relationships or use conventional deep learning models like CNN and LSTM, which do not explicitly learn inter-variable correlations. In this study, we propose a novel unsupervised anomaly detection method that integrates Sparse Autoencoder with Graph Transformer network (SGTrans). SGTrans leverages Sparse Autoencoder for the dimensionality reduction and reconstruction of high-dimensional time series, thus extracting meaningful hidden representations. Then, the multivariate time series are mapped into a graph structure. We introduce a multi-head attention mechanism from Transformer into graph structure learning, constructing a Graph Transformer network forecasting module. This module performs attentive information propagation between long-distance sensor nodes and explicitly models the complex temporal dependencies among them to enhance the prediction of future behaviors. Extensive experiments and evaluations on three publicly available real-world datasets demonstrate the effectiveness of our approach.

Enhancing Anomaly Detection in Multivariate Time Series with Stacked Transformer Encoders and Adaptive Positional Embeddings

A two-stage adversarial Transformer based approach for multivariate industrial time series anomaly detection

Time Series Anomaly Detection with a Transformer Residual Autoencoder-Decoder.

EdgeConvFormer: Dynamic Graph CNN and Transformer based Anomaly Detection in Multivariate Time Series

Transformer-based multivariate time series anomaly detection using inter-variable attention mechanism

StackVAE-G: an Efficient and Interpretable Model for Time Series Anomaly Detection

Variational transformer-based anomaly detection approach for multivariate time series

SiET: Spatial information enhanced transformer for multivariate time series anomaly detection

Graph Transformer Network Incorporating Sparse Representation for Multivariate Time Series Anomaly Detection

Implementation of a Sequence-to-Sequence Stacked Sparse Long Short-Term Memory Autoencoder for Anomaly Detection on Multivariate Timeseries Data of Industrial Blower Ball Bearing Units

Segmentation-Based Adversarial Denoising Auto-Encoder for Anomaly Detection in Multivariate Time Series Data

Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy

SA2E-AD: A Stacked Attention Autoencoder for Anomaly Detection in Multivariate Time Series

Concept Drift Adaptation for Time Series Anomaly Detection via Transformer

LGAT: A Novel Model for Multivariate Time Series Anomaly Detection with Imporved Anomaly Transformer and Learning Graph Structures

An Enhancing Timeseries Anomaly Detection Using LSTM and Bi-LSTM Architectures

HybridAD: A Hybrid Model-Driven Anomaly Detection Approach for Multivariate Time Series

Anomaly detection in surveillance videos using Transformer with margin learning

Multivariate Time Series Anomaly Detection with Fourier Time Series Transformer

MEMTO: Memory-guided Transformer for Multivariate Time Series Anomaly Detection

TCF-Trans: Temporal Context Fusion Transformer for Anomaly Detection in Time Series