Efficient Log Anomaly Detection Based on Dimension Reduction and Attention Aware TCN

Zhihao Xu,Yuliang Shi,Zhiyuan Su,Li Song,Jianjun Zhang,Xinjun Wang,Hui Li
DOI: https://doi.org/10.1007/978-981-97-2303-4_33
2024-01-01
Abstract:The software system usually records important runtime information in the log for troubleshooting. Researchers mine large log data for anomalies. Many studies use log data to build deep-learning models for detecting system anomalies. Although progress has been made in log anomaly detection on high-performance computing platforms, it is still difficult to achieve real-time and accurate anomaly detection on mobile devices and Internet of Things devices, as these devices usually do not have high computational power. To solve the above limitations, we propose an efficient log anomaly detection based on dimension reduction and attention aware temporal convolutional network method, namely EfficientLog. The model achieves efficient and accurate log detection in two ways: (1) it reduces the communication cost between mobile devices and cloud computing platforms by reducing the log vector dimension through BERT-whitening, and (2) it detects log anomalies by using the attention aware temporal convolutional network to reduce model testing time and computational consumption. We evaluate the proposed method on two public datasets, and the experimental results show that EfficientLog can outperform existing popular log-based anomaly detection methods in terms of detection accuracy and computational consumption.
What problem does this paper attempt to address?