Efficient Multivariate Time Series Anomaly Detection Through Transfer Learning for Large-Scale Web Services

Yongqian Sun,Minghan Liang,Zeyu Che,Dongwen Li,Tinghua Zheng,Shenglin Zhang,Pengtian Zhu,Yuzhi Zhang,Dan Pei
DOI: https://doi.org/10.1109/icws60048.2023.00033
IF: 3.685
2024-01-01
ACM Transactions on Software Engineering and Methodology
Abstract:Timely anomaly detection of multivariate time series (MTS) is of vital importance for managing large-scale software systems. However, many deep learning-based MTS anomaly detection models require long-term MTS training data to achieve optimal performance, which often conflicts with the frequent pattern changes observed in software systems. Moreover, the training overhead of vast MTS in large-scale software systems is unacceptably high. To address these issues, we design OmniTransfer, a model-agnostic framework that combines weighted hierarchical agglomerative clustering with an adaptive transfer learning strategy, making many state-of-the-art (SOTA) MTS anomaly detection models efficient and effective. Extensive experiments using real-world data from a large web content service provider and a network operator show that OmniTransfer significantly reduces the model initialization time by 46.49% and the training cost by 74.51%, while maintaining high accuracy in detecting anomalies.
What problem does this paper attempt to address?