Abstract:Most enterprise applications use logging as a mechanism to diagnose anomalies, which could help with reducing system downtime. Anomaly detection using software execution logs has been explored in several prior studies, using both classical and deep neural network-based machine learning models. In recent years, the research has largely focused in using variations of sequence-based deep neural networks (e.g., Long-Short Term Memory and Transformer-based models) for log-based anomaly detection on open-source data. However, they have not been applied in industrial datasets, as often. In addition, the studied open-source datasets are typically very large in size with logging statements that do not change much over time, which may not be the case with a dataset from an industrial service that is relatively new. In this paper, we evaluate several state-of-the-art anomaly detection models on an industrial dataset from our research partner, which is much smaller and loosely structured than most large scale open-source benchmark datasets. Results show that while all models are capable of detecting anomalies, certain models are better suited for less-structured datasets. We also see that model effectiveness changes when a common data leak associated with a random train-test split in some prior work is removed. A qualitative study of the defects' characteristics identified by the developers on the industrial dataset further shows strengths and weaknesses of the models in detecting different types of anomalies. Finally, we explore the effect of limited training data by gradually increasing the training set size, to evaluate if the model effectiveness does depend on the training set size.

Deep Neural Network Based Log Analysis

Log Analysis For Network Attack Detection Using Deep Learning Models

Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection

Log2NS: Enhancing Deep Learning Based Analysis of Logs With Formal to Prevent Survivorship Bias

DeepUserLog: Deep Anomaly Detection on User Log Using Semantic Analysis and Key-Value Data

Natural Language Processing-based Model for Log Anomaly Detection

OneLog: Towards End-to-End Training in Software Log Anomaly Detection

Distributed system anomaly detection using deep learning‐based log analysis

OneLog: towards end-to-end software log anomaly detection

LogGD:Detecting Anomalies from System Logs by Graph Neural Networks

DLLog: An Online Log Parsing Approach for Large-Scale System

A Security Log Analysis Scheme Using Deep Learning Algorithm for IDSs in Social Network

Pinpointing Anomaly Events in Logs from Stability Testing -- N-Grams vs. Deep-Learning

Deep Learning for Anomaly Detection in Log Data: A Survey

Log-based Anomaly Detection Without Log Parsing

Machine Learning in Business Process Monitoring: A Comparison of Deep Learning and Classical Approaches Used for Outcome Prediction

Log-based Anomaly Detection of Enterprise Software: An Empirical Study

MLog: Mogrifier LSTM-based Log Anomaly Detection Approach Using Semantic Representation

Systematic Evaluation of Deep Learning Models for Log-based Failure Prediction

DeepTraLog: Trace-Log Combined Microservice Anomaly Detection through Graph-based Deep Learning