Abstract:Cyber–physical systems (CPSs) serve as the pivotal core of Internet of Things (IoT) infrastructures, such as smart grids and intelligent transportation, deploying interconnected sensing devices to monitor operating status. With increasing decentralization, the surge in sensor devices expands the potential vulnerability to cyber attacks. It is imperative to conduct anomaly detection research on the multivariate time series data that these sensors produce to bolster the security of distributed CPSs. However, the high dimensionality, absence of anomaly labels in real-world datasets, and intricate non-linear relationships among sensors present considerable challenges in formulating effective anomaly detection algorithms. Recent deep-learning methods have achieved progress in the field of anomaly detection. Yet, many methods either rely on statistical models that struggle to capture non-linear relationships or use conventional deep learning models like CNN and LSTM, which do not explicitly learn inter-variable correlations. In this study, we propose a novel unsupervised anomaly detection method that integrates Sparse Autoencoder with Graph Transformer network (SGTrans). SGTrans leverages Sparse Autoencoder for the dimensionality reduction and reconstruction of high-dimensional time series, thus extracting meaningful hidden representations. Then, the multivariate time series are mapped into a graph structure. We introduce a multi-head attention mechanism from Transformer into graph structure learning, constructing a Graph Transformer network forecasting module. This module performs attentive information propagation between long-distance sensor nodes and explicitly models the complex temporal dependencies among them to enhance the prediction of future behaviors. Extensive experiments and evaluations on three publicly available real-world datasets demonstrate the effectiveness of our approach.

DMU-TransNet: Dense multi-scale U-shape transformer network for anomaly detection

TransUNetCD: A Hybrid Transformer Network for Change Detection in Optical Remote-Sensing Images

Multiresolution feature guidance based transformer for anomaly detection

UTRAD: Anomaly detection and localization with U-Transformer

MTC-NET: A Multi-Channel Independent Anomaly Detection Method for Network Traffic

DDMT: Denoising Diffusion Mask Transformer Models for Multivariate Time Series Anomaly Detection

Hierarchical Vector Quantized Transformer for Multi-class Unsupervised Anomaly Detection

HmsU-Net: A hybrid multi-scale U-net based on a CNN and transformer for medical image segmentation

DTAAD: Dual Tcn-attention networks for anomaly detection in multivariate time series data

TDS-Net: Transformer Enhanced Dual-Stream Network for Video Anomaly Detection

ITran: A novel transformer-based approach for industrial anomaly detection and localization

TUT: Template-Augmented U-Net Transformer for Unsupervised Anomaly Detection

TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation

MSCT-UNET: multi-scale contrastive transformer within U-shaped network for medical image segmentation

UCTNet: Uncertainty-guided CNN-Transformer hybrid networks for medical image segmentation

Graph Transformer Network Incorporating Sparse Representation for Multivariate Time Series Anomaly Detection

Self-Supervised Masked Convolutional Transformer Block for Anomaly Detection

BTAD: A binary transformer deep neural network model for anomaly detection in multivariate time series data

DUdetector: A Dual-Granularity Unsupervised Model for Network Anomaly Detection

ADTR: Anomaly Detection Transformer with Feature Reconstruction