Abstract:Abstract The orchestration of cloud computing infrastructures is challenging, considering the number, heterogeneity and dynamicity of the involved resources, along with the highly distributed nature of the applications that use them for computation and storage. Evidently, the volume of relevant monitoring data can be significant, and the ability to collect, analyze, and act on this data in real time is critical for the infrastructure’s efficient use. In this study, we introduce a novel methodology that adeptly manages the diverse, dynamic, and voluminous nature of cloud resources and the applications that they support. We use knowledge graphs to represent computing and storage resources and illustrate the relationships between them and the applications that utilize them. We then train GraphSAGE to acquire vector-based representations of the infrastructures’ properties, while preserving the structural properties of the graph. These are efficiently provided as input to two unsupervised machine learning algorithms, namely CBLOF and Isolation Forest, for the detection of storage and computing overusage events, where CBLOF demonstrates better performance across all our evaluation metrics. Following the detection of such events, we have also developed appropriate re-optimization mechanisms that ensure the performance of the served applications. Evaluated in a simulated environment, our methods demonstrate a significant advancement in anomaly detection and infrastructure optimization. The results underscore the potential of this closed-loop operation in dynamically adapting to the evolving demands of cloud infrastructures. By integrating data representation and machine learning methods with proactive management strategies, this research contributes substantially to the field of cloud computing, offering a scalable, intelligent solution for modern cloud infrastructures.

An Anomaly-based Detection System for Monitoring Kubernetes Infrastructures

Detection of Cluster Anomalies With ML Techniques

Anomaly detection in the CERN cloud infrastructure

A novel approach to feature collection for anomaly detection in Kubernetes environment and agent for metrics collection from Kubernetes nodes

KubAnomaly: Anomaly detection for the Docker orchestration platform with neural network approaches

Performance Analysis of Anomaly Detection Methods for Application System on Kubernetes with Auto-scaling and Self-healing

Anomaly Detection in Cloud Computing using Knowledge Graph Embedding and Machine Learning Mechanisms

Development of infrastructure for anomalies detectionin big data

Anomaly detection in cloud environment using artificial intelligence techniques

Anomaly Detection with Artificial Intelligence Methods : An Overview

Anomaly Detection for Industrial Big Data

Anomaly Detection for Incident Response at Scale

An ensemble-based framework for user behaviour anomaly detection and classification for cybersecurity

Intelligent Anomaly Detection and Mitigation in Data Centers

Extending Isolation Forest for Anomaly Detection in Big Data via K-Means

Develop End-to-End Anomaly Detection System

Anomaly detection of unstructured big data via semantic analysis and dynamic knowledge graph construction

Rapid Deployment of Anomaly Detection Models for Large Number of Emerging KPI Streams.

A Methodological Report on Anomaly Detection on Dynamic Knowledge Graphs

Advancing Software Security and Reliability in Cloud Platforms through AI-based Anomaly Detection