HEDGE: Heterogeneous Semantic Dynamic Graph Framework for Log Anomaly Detection in Digital Service Network

Bohao Qian,Mengying Zhu,Mengyuan Yang,Enze Wu,Guojie Xie,Yuebing Liang,Xiaolin Zheng
DOI: https://doi.org/10.1109/icws62655.2024.00041
2024-01-01
Abstract:Log anomaly detection in digital service networks is challenging due to the heterogeneity and complexity of log formats and semantics. Traditional log anomaly detection methods struggle with two main challenges: the inability to directly correlate heterogeneous logs and the semantic heterogeneity across and within logs. To address these challenges, we propose a novel framework, HEDGE, which constructs a dynamic heterogeneous log graph to capture spatio-temporal relationships between logs, reflecting fine-grained semantic correlations and evolutionary properties of sequential logs comprehensively and detecting log anomalies effectively. To capture log representations under heterogeneity from both semantic and spatio-temporal perspectives, HEDGE not only pre-trains a dual-tower SemanticFormer based on BERT to align global and local semantic information for heterogeneous nodes but also adopts a dynamic heterogeneous graph model to learn spatio-temporal topological features within inner-snapshot and intra-snapshot contexts. Extensive experiments on public datasets demonstrate the superiority of our framework compared to state-of-the-art baselines.
What problem does this paper attempt to address?