Abstract:Logs are widely used in the development and maintenance of software systems. Logs can help engineers understand the runtime behavior of systems and diagnose system failures. For anomaly diagnosis, existing methods generally use log event data extracted from historical logs to build diagnostic models. However, we find that existing methods do not make full use of two types of features, (1) statistical features: some inherent statistical features in log data, such as word frequency and abnormal label distribution, are not well exploited. Compared with log raw data, statistical features are deterministic and naturally compatible with corresponding tasks. (2) semantic features: Logs contain the execution logic behind software systems, thus log statements share deep semantic relationships. How to effectively combine statistical features and semantic features in log data to improve the performance of log anomaly diagnosis is the key point of this paper. In this paper, we propose an adaptive semantic gate networks (ASGNet) that combines statistical features and semantic features to selectively use statistical features to consolidate log text semantic representation. Specifically, ASGNet encodes statistical features via a variational encoding module and fuses useful information through a well-designed adaptive semantic threshold mechanism. The threshold mechanism introduces the information flow into the classifier based on the confidence of the semantic features in the decision, which is conducive to training a robust classifier and can solve the overfitting problem caused by the use of statistical features. The experimental results on the real data set show that our method proposed is superior to all baseline methods in terms of various performance indicators.

Research on System Log Anomaly Detection Combining Two-way Slice GRU and GA-Attention Mechanism

LogAnomaly: Unsupervised Detection of Sequential and Quantitative Anomalies in Unstructured Logs

Experience Report: Deep Learning-based System Log Analysis for Anomaly Detection

Detecting Log Anomalies with Multi-Head Attention (LAMA)

An Anomaly Detection Approach of Part-of-Speech Log Sequence Via Population Based Training

Big-data Analysis of Multi-Source Logs for Anomaly Detection on Network-Based System.

LogGD:Detecting Anomalies from System Logs by Graph Neural Networks

A LSTM-Based Anomaly Detection Model for Log Analysis

Log2graphs: An Unsupervised Framework for Log Anomaly Detection with Efficient Feature Extraction

LogPS: A Robust Log Sequential Anomaly Detection Approach Based on Natural Language Processing

ASGNet: Adaptive Semantic Gate Networks for Log-Based Anomaly Diagnosis

Natural Language Processing-based Model for Log Anomaly Detection

An Integrated Method for Anomaly Detection From Massive System Logs.

Log-based Anomaly Detection based on EVT Theory with feedback

GLAD: Content-aware Dynamic Graphs For Log Anomaly Detection

LogAttn: Unsupervised Log Anomaly Detection with an AutoEncoder Based Attention Mechanism

LogOnline: A Semi-Supervised Log-Based Anomaly Detector Aided with Online Learning Mechanism.

LogRAG: Semi-Supervised Log-based Anomaly Detection with Retrieval-Augmented Generation

Distributed system anomaly detection using deep learning‐based log analysis

Log Sequence Anomaly Detection Method Based on Contrastive Adversarial Training and Dual Feature Extraction

Anomaly Detection Model for Log Based on LSTM Network and Variational Autoencoder