Abstract:Most enterprise applications use logging as a mechanism to diagnose anomalies, which could help with reducing system downtime. Anomaly detection using software execution logs has been explored in several prior studies, using both classical and deep neural network-based machine learning models. In recent years, the research has largely focused in using variations of sequence-based deep neural networks (e.g., Long-Short Term Memory and Transformer-based models) for log-based anomaly detection on open-source data. However, they have not been applied in industrial datasets, as often. In addition, the studied open-source datasets are typically very large in size with logging statements that do not change much over time, which may not be the case with a dataset from an industrial service that is relatively new. In this paper, we evaluate several state-of-the-art anomaly detection models on an industrial dataset from our research partner, which is much smaller and loosely structured than most large scale open-source benchmark datasets. Results show that while all models are capable of detecting anomalies, certain models are better suited for less-structured datasets. We also see that model effectiveness changes when a common data leak associated with a random train-test split in some prior work is removed. A qualitative study of the defects' characteristics identified by the developers on the industrial dataset further shows strengths and weaknesses of the models in detecting different types of anomalies. Finally, we explore the effect of limited training data by gradually increasing the training set size, to evaluate if the model effectiveness does depend on the training set size.

Augmenting Log-based Anomaly Detection Models to Reduce False Anomalies with Human Feedback

Feedback-Aware Anomaly Detection Through Logs for Large-Scale Software Systems

Hilogx: noise-aware log-based anomaly detection with human feedback

Reducing Events to Augment Log-based Anomaly Detection Models: An Empirical Study

AFALog: A General Augmentation Framework for Log-based Anomaly Detection with Active Learning

Natural Language Processing-based Model for Log Anomaly Detection

AcLog: an Approach to Detecting Anomalies from System Logs with Active Learning

LogCAE: an Approach for Log-based Anomaly Detection with Active Learning and Contrastive Learning

Log-based Anomaly Detection of Enterprise Software: An Empirical Study

Log-based Anomaly Detection based on EVT Theory with feedback

Log-based Anomaly Detection with Deep Learning: How Far Are We?

MetaLog: Generalizable Cross-System Anomaly Detection from Logs with Meta-Learning.

Enhancing Anomaly Detection Models for Industrial Applications through SVM-Based False Positive Classification

From Explanation to Action: An End-to-End Human-in-the-loop Framework for Anomaly Reasoning and Management

Machine Learning Based Anomaly Detection of Log Files Using Ensemble Learning and Self-Attention

Brief Announcement: Automatic Log Enhancement for Fault Diagnosis.

A Reliable Framework for Human-in-the-Loop Anomaly Detection in Time Series

LogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs

Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection

Research of Anomaly detection based on Dynamic Anomaly Detection Enhancement Framework

OMLog: Online Log Anomaly Detection for Evolving System with Meta-learning