Abstract:Logs are prevalent in modern cloud systems and serve as a valuable source of information for system maintenance. Over the years, a lot of research and industrial efforts have been devoted to the field of log-based anomaly detection. Through analyzing the limitations of existing approaches, we find that most of them still suffer from practical issues and are thus hard to be applied in real-world scenarios. For example, supervised approaches are dependent on a large amount of labeled log data for training, which can require much manual labeling effort. Besides, log instability, which is a pervasive issue in real-world systems, poses great challenge to existing methods, especially under the presence of many dissimilar new log events. To overcome these problems, we propose LogOnline, which is a semi supervised anomaly detector aided with online learning mechanism. The semi-supervised nature of LogOnline makes it able to get rid of the erroneous and time-consuming manual labeling of log data. Based on our proposed online learning mechanism, LogOnline can learn the normal sequence patterns continuously as new log sequences emerge, thus staying robust to unstable log data. Unlike previous works, the proposed online learning mechanism requires no labeled log data nor human intervention in the process. We have evaluated LogOnline on two widely used public datasets, and the experimental results demonstrate the effectiveness of LogOnline. In particular, LogOnline achieves a comparable result with the studied supervised approaches, outperforming all semi-supervised counterparts. When the log instability issue is more common, LogOnline exhibits the best performance over all compared approaches, further confirming its practicability.

Semi-supervised log anomaly detection based on bidirectional temporal convolution network

ETCNLog: A System Log Anomaly Detection Method Based on Efficient Channel Attention and Temporal Convolutional Network

An Anomaly Detection Approach of Part-of-Speech Log Sequence Via Population Based Training

SemLog: A Semantics-based Approach for Anomaly Detection in Big Data System Logs

CLDTLog: System Log Anomaly Detection Method Based on Contrastive Learning and Dual Objective Tasks

Log Anomaly Detection method based on BERT model optimization

LogBD: A Log Anomaly Detection Method Based on Pretrained Models and Domain Adaptation

Natural Language Processing-based Model for Log Anomaly Detection

An Anomaly Detection Approach Based on Bidirectional Temporal Convolutional Network and Multi-Head Attention Mechanism

CANet: Convolutional Attention-based Network for Multivariate Time Series Anomaly Detection

Research on Log Anomaly Detection Based on Sentence-BERT

LogBERT: Log Anomaly Detection via BERT

LogPS: A Robust Log Sequential Anomaly Detection Approach Based on Natural Language Processing

BERT-Log: Anomaly Detection for System Logs Based on Pre-trained Language Model

LogCSS: Log anomaly detection based on BERT-CNN with context-semantics-statistics features

LogST: Log Semi-supervised Anomaly Detection Based on Sentence-BERT

DualAttlog: Context aware dual attention networks for log-based anomaly detection

ConAnomaly: Content-Based Anomaly Detection for System Logs

LogOnline: A Semi-Supervised Log-Based Anomaly Detector Aided with Online Learning Mechanism.

Log Sequence Anomaly Detection Method Based on Contrastive Adversarial Training and Dual Feature Extraction

Log-based Anomaly Detection Without Log Parsing