Abstract:The microservices architecture is extensively utilized in cloud-based application development, characterized by the construction of applications through a series of functionally independent, small, autonomous services. This architectural approach is renowned for its attributes such as high cohesion, availability, low coupling, and exceptional scalability. The detection of runtime system point anomalies in microservices architectures is crucial for enhancing the Quality of Service(QoS). Furthermore, identifying the classes of detected anomalies is critical in practical applications. However, given the highly dynamic nature of microservices systems as a distributed computing architecture, conducting real-time system anomaly detection on distributed independent microservices poses a challenging task. To address these challenges, we propose the System Anomaly Detection and Multi-Classification based on Multi-Task Feature Fusion Federated Learning (SADMC-MT-FF-FL) framework. Initially, we introduce a distributed learning framework based on Multi-task Federated Learning (MT-FL) to construct multi-classification anomaly detection models for each microservice. Secondly, to identify complex system anomaly patterns and features during the runtime of microservices, we develop a feature extractor based on External Attention Mechanism and Multi-channel Residual Structure (EA-MRS). Finally, we design a Local–Global Feature-based Parallel Knowledge Transfer (LGF-PKT) framework, utilizing parallel knowledge transfer to parallelize weight updates for local and global features. To validate the effectiveness of our approach, we conducted comprehensive comparative experiments on the microservices benchmark platforms Sock-Shop and Train-Ticket. The experimental results on anomaly detection for multiclassification systems demonstrate that SADMC-MT-FF-FL outperforms the best baseline method by 28.3% and 27.8% for Macro F1 and Micro F1 on Train-Ticket, and by 8.8% and 8.6% on Sock-Shop, respectively. Additionally, we conducted comparison experiments on three public datasets, SWaT, SMD, and SKAB. The F1 scores were 0.5% higher than those of the centralized methods on SMD, respectively, 6% and 2.8% higher than those of the federated learning based method on SWaT and SKAB. Source codes are available at: https://github.com/icc-lab-xhu1/SADMC-MT-FF-FL.

Robust System Instance Clustering for Large-Scale Web Services

Efficient Multivariate Time Series Anomaly Detection Through Transfer Learning for Large-Scale Web Services

Log clustering based problem identification for online service systems

Robust Anomaly Clue Localization of Multi-Dimensional Derived Measure for Online Video Services

Matrix Factorization and Deep Autoencoder based Clustering Scheme for Large-scale UAV Networks

Practical Anomaly Detection over Multivariate Monitoring Metrics for Online Services

Efficient and Robust KPI Outlier Detection for Large-Scale Datacenters

Efficient KPI Anomaly Detection Through Transfer Learning for Large-Scale Web Services

An Adaptive Model For Building Service-Partition System

Robust Anomaly Detection for Multivariate Time Series Through Stochastic Recurrent Neural Network

MonitorAssistant: Simplifying Cloud Service Monitoring Via Large Language Models

Fine Clustering Analysis of Internet Financial Credit Investigation Based on Big Data

Enhancing Cluster Resilience: LLM-agent Based Autonomous Intelligent Cluster Diagnosis System and Evaluation Framework

A Web Service Clustering Method Based on Semantic Similarity and Multidimensional Scaling Analysis

A hybrid unsupervised clustering-based anomaly detection method

Log-Based Anomaly Detection With Robust Feature Extraction and Online Learning

Cluster-Wide Task Slowdown Detection in Cloud System

Multi-task federated learning-based system anomaly detection and multi-classification for microservices architecture

Robust Dual Clustering with Adaptive Manifold Regularization

Anomaly detecting and ranking of the cloud computing platform by multi-view learning

Time Series Based Killer Task Online Recognition Service: A Google Cluster Case Study