Abstract:Anomaly diagnosis relying on system logs to record runtime events is essential for improving the service quality of distributed systems and reducing economic losses. However, most existing log-based anomaly detection approaches depend on the assumption of the fixed quantitative or sequential patterns of a normal log event sequence. This assumption is challenged in the context of practical distributed and parallel systems due to the dynamic pattern of the log sequence, log data noise as well as concurrency of multiple anomalies. Against these challenges, this paper aims to perform the robust log-based anomaly diagnosis by capturing the event context information of the log event graph, called LogGraph, instead of straightforwardly employing the fixed quantitative or sequential patterns of the log records, thus reducing its sensitivity to the log flaws and the concurrency of multiple anomalies. Specifically, in order to handle multiple anomalies concurrency, LogGraph invokes the association rule to decouple log sequences. It further reinterprets a log record sequence into a log event graph modeled by event semantic embedding and event adjacency matrix. An attention-based Gated Graph Neural Network (GGNN) model is developed to capture the semantic information of the log graph, which enables the fine-grained and robust anomaly identification of the proposed scheme. We use real log data sets collected from Hadoop systems and network switches to verify the effectiveness of the proposed LogGraph in log data scenarios that contain noise and multiple anomalies concurrency problems. The experimental results show that the proposed LogGraph achieves high performance and strong robustness in anomaly diagnosis.

A Framework for Human-Centered Exploration of Complex Event Log Graphs

Multilayer Block Models for Exploratory Analysis of Computer Event Logs

A Framework for Pattern Mining and Anomaly Detection in Multi-dimensional Time Series and Event Logs

VloGraph: A Virtual Knowledge Graph Framework for Distributed Security Log Analysis

A unified framework for multi-level analysis of distributed learning

Multi-Dimensional Event Data in Graph Databases

Holistic Processing and Exploring Event Logs

GECCO: Constraint-driven Abstraction of Low-level Event Logs

Virtual Knowledge Graphs for Federated Log Analysis

Frequent Pattern-based Graph Exploration

Visual Analysis of Complex Temporal Networks Supported by Analytic Provenance

Graph Exploration with Embedding-Guided Layouts

GLog: A high level graph analysis system using MapReduce

LogGraph: Log Event Graph Learning Aided Robust Fine-Grained Anomaly Diagnosis

GraphLDA : Latent Dirichlet Allocation-based Visual Exploration of Dynamic Graphs

Process Trace Querying using Knowledge Graphs and Notation3

Exploring and mining attributed sequences of interactions

A Graphical Representation for Identifier Structure in Logs

Unified Semantic Log Parsing and Causal Graph Construction for Attack Attribution

Experience Report: Log Mining Using Natural Language Processing and Application to Anomaly Detection