Multivariate Time Series Anomaly Detection Based on Pre-trained Models with Dual-Attention Mechanism

Yongqian Sun,Yang Guo,Minghan Liang,Xidao Wen,Junhua Kuang,Shenglin Zhang,Hongbo Li,Kaixu Xia,Dan Pei
DOI: https://doi.org/10.1109/issrew63542.2024.00050
2024-01-01
Abstract:In major tech companies, monitoring server performance data with anomaly detection algorithms is crucial for assessing operational status. Existing models often require separate training or fine-tuning for each server due to generalization limitations, leading to increased storage, memory, and training costs. As the number of servers grows, this approach becomes impractical. To address this, we propose using pretrained language models for time series anomaly detection, leveraging their strong generalization capabilities. Specifically, we employ two pre-trained GPT-2 models as backbones and implement a two-stage fine-tuning strategy to retain learned knowledge while adapting to specific business data characteristics. Our experiments on multiple anomaly detection datasets demonstrate that our method achieves the best average F1-Score, outperforming the leading baseline by 7%.
What problem does this paper attempt to address?