Abstract:Cyber–physical systems (CPSs) serve as the pivotal core of Internet of Things (IoT) infrastructures, such as smart grids and intelligent transportation, deploying interconnected sensing devices to monitor operating status. With increasing decentralization, the surge in sensor devices expands the potential vulnerability to cyber attacks. It is imperative to conduct anomaly detection research on the multivariate time series data that these sensors produce to bolster the security of distributed CPSs. However, the high dimensionality, absence of anomaly labels in real-world datasets, and intricate non-linear relationships among sensors present considerable challenges in formulating effective anomaly detection algorithms. Recent deep-learning methods have achieved progress in the field of anomaly detection. Yet, many methods either rely on statistical models that struggle to capture non-linear relationships or use conventional deep learning models like CNN and LSTM, which do not explicitly learn inter-variable correlations. In this study, we propose a novel unsupervised anomaly detection method that integrates Sparse Autoencoder with Graph Transformer network (SGTrans). SGTrans leverages Sparse Autoencoder for the dimensionality reduction and reconstruction of high-dimensional time series, thus extracting meaningful hidden representations. Then, the multivariate time series are mapped into a graph structure. We introduce a multi-head attention mechanism from Transformer into graph structure learning, constructing a Graph Transformer network forecasting module. This module performs attentive information propagation between long-distance sensor nodes and explicitly models the complex temporal dependencies among them to enhance the prediction of future behaviors. Extensive experiments and evaluations on three publicly available real-world datasets demonstrate the effectiveness of our approach.

SERT: A transfomer based model for multivariate temporal sensor data with missing values for environmental monitoring

SERT: A Transfomer Based Model for Spatio-Temporal Sensor Data with Missing Values for Environmental Monitoring

Using Temporal Correlation and Time Series to Detect Missing Activity-Driven Sensor Events.

Spatiotemporal Transformer for Imputing Sparse Data: A Deep Learning Approach

SiET: Spatial information enhanced transformer for multivariate time series anomaly detection

ALERT-Transformer: Bridging Asynchronous and Synchronous Machine Learning for Real-Time Event-based Spatio-Temporal Data

Missing Value Imputation of Wireless Sensor Data for Environmental Monitoring

TENT: Tensorized Encoder Transformer for Temperature Forecasting

Graph Transformer Network Incorporating Sparse Representation for Multivariate Time Series Anomaly Detection

Unsupervised Spatio-Temporal State Estimation for Fine-grained Adaptive Anomaly Diagnosis of Industrial Cyber-physical Systems

Towards efficient similarity embedded temporal Transformers via extended timeframe analysis

Time Series Representation Models

ImputeFormer: Low Rankness-Induced Transformers for Generalizable Spatiotemporal Imputation

Generalizability Under Sensor Failure: Tokenization + Transformers Enable More Robust Latent Spaces

A spatio-temporal LSTM model to forecast across multiple temporal and spatial scales

Stecformer: Spatio-temporal Encoding Cascaded Transformer for Multivariate Long-term Time Series Forecasting

Recover Missing Sensor Data with Iterative Imputing Network

Earthfarsser: Versatile Spatio-Temporal Dynamical Systems Modeling in One Model

Toward Resilient Electric Vehicle Charging Monitoring Systems: Curriculum Guided Multi-Feature Fusion Transformer

Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observation

STARS: Sensor-agnostic Transformer Architecture for Remote Sensing