Abstract:As Internet applications continue to scale up, microservice architecture has become increasingly popular due to its flexibility and logical structure. Anomaly detection in traces that record inter-microservice invocations is essential for diagnosing system failures. Deep learning-based approaches allow for accurate modeling of structural features (i.e., call paths) and latency features (i.e., call response time), which can determine the anomaly of a particular trace sample. However, the point-wise manner employed by these methods results in substantial system detection overhead and impracticality, given the massive volume of traces (billion-level). Furthermore, the point-wise approach lacks high-level information, as identical sub-structures across multiple traces may be encoded differently. In this paper, we introduce the first Group-wise Trace anomaly detection algorithm, named GTrace. This method categorizes the traces into distinct groups based on their shared sub-structure, such as the entire tree or sub-tree structure. A group-wise Variational AutoEncoder (VAE) is then employed to obtain structural representations. Moreover, the innovative "predicting latency with structure" learning paradigm facilitates the association between the grouped structure and the latency distribution within each group. With the group-wise design, representation caching, and batched inference strategies can be implemented, which significantly reduces the burden of detection on the system. Our comprehensive evaluation reveals that GTrace outperforms state-of-the-art methods in both performances (2.64% to 195.45% improvement in AUC metrics and 2.31% to 40.92% improvement in best F-Score) and efficiency (21.9x to 28.2x speedup). We have deployed and assessed the proposed algorithm on eBay's microservices cluster, and our code is available at https://github.com/NetManAIOps/GTrace.git.

DeepTraLog: Trace-Log Combined Microservice Anomaly Detection through Graph-based Deep Learning

BertHTLG: Graph-Based Microservice Anomaly Detection Through Sentence-Bert Enhancement.

TraceGra: A Trace-Based Anomaly Detection for Microservice Using Graph Deep Learning

Unsupervised Detection of Microservice Trace Anomalies Through Service-Level Deep Bayesian Networks

From Point-wise to Group-wise: A Fast and Accurate Microservice Trace Anomaly Detection Approach

Unsupervised Anomaly Detection on Microservice Traces through Graph VAE

Efficient and Robust Trace Anomaly Detection for Large-Scale Microservice Systems

Microservice Anomaly Detection Based on Tracing Data Using Semi-supervised Learning

ServiceAnomaly: An anomaly detection approach in microservices using distributed traces and profiling metrics

Large-Scale Trace Analysis for Microservice Anomaly Detection and Root Cause Localization

Twin Graph-based Anomaly Detection via Attentive Multi-Modal Learning for Microservice System

Deep Attentive Anomaly Detection for Microservice Systems with Multimodal Time-Series Data

TraceStream: Anomalous Service Localization Based on Trace Stream Clustering with Online Feedback

Anomaly detection in microservice environments using distributed tracing data analysis and NLP

Graph-based trace analysis for microservice architecture understanding and problem diagnosis

An Approach for Anomaly Diagnosis Based on Hybrid Graph Model with Logs for Distributed Services

An Intelligent Anomaly Detection Scheme for Micro-Services Architectures with Temporal and Spatial Data Analysis.

Network-Centric Distributed Tracing with DeepFlow: Troubleshooting Your Microservices in Zero Code.

Root-Cause Metric Location for Microservice Systems Via Log Anomaly Detection

Distributed system anomaly detection using deep learning‐based log analysis

Enjoy your observability: an industrial survey of microservice tracing and analysis