Abstract:Cloud service owners need to continuously monitor their services to ensure high availability and reliability. Gaps in monitoring can lead to delay in incident detection and significant negative customer impact. Current process of monitor creation is ad-hoc and reactive in nature. Developers create monitors using their tribal knowledge and, primarily, a trial and error based process. As a result, monitors often have incomplete coverage which leads to production issues, or, redundancy which results in noise and wasted effort. In this work, we address this issue by proposing an intelligent monitoring framework that recommends monitors for cloud services based on their service properties. We start by mining the attributes of 30,000+ monitors from 791 production services at Microsoft and derive a structured ontology for monitors. We focus on two crucial dimensions: what to monitor (resources) and which metrics to monitor. We conduct an extensive empirical study and derive key insights on the major classes of monitors employed by cloud services at Microsoft, their associated dimensions, and the interrelationship between service properties and this ontology. Using these insights, we propose a deep learning based framework that recommends monitors based on the service properties. Finally, we conduct a user study with engineers from Microsoft which demonstrates the usefulness of the proposed framework. The proposed framework along with the ontology driven projections, succeeded in creating production quality recommendations for majority of resource classes. This was also validated by the users from the study who rated the framework's usefulness as 4.27 out of 5.

MonValley: an Unified Monitoring and Management Framework for Cloud Services

SimMon: A Toolkit for Simulating Monitoring Mechanism in Cloud Computing Environments.

A Runtime Model Based Monitoring Approach for Cloud

SimMon: a toolkit for simulation of monitoring mechanisms in cloud computing environment.

JTangCMS: an Efficient Monitoring System for Cloud Platforms.

HiSML: A High-Level Integrated Service Monitoring Language.

A simulation cloud monitoring framework and its evaluation model.

Intelligent Monitoring Framework for Cloud Services: A Data-Driven Approach

Managing Clouds in Cloud Platforms

Cloud Service Monitoring System Based on SLA

MonitorAssistant: Simplifying Cloud Service Monitoring Via Large Language Models

An Online Monitoring Approach for Web Service Requirements

An efficient data dissemination approach for cloud monitoring

A Multi-Source Data Oriented Framework of Vehicle Monitoring System

Cloud Data Monitoring Management and Visual Application System Based on Spring Boot

Model Based Monitoring and Controlling for Platform-as-a-Service PaaS

MonALISA : A Distributed Monitoring Service Architecture

A model-based framework for platform management in Cloud.

A Wsla-Based Monitoring System For Grid Service-Gsmon

Zero+: Monitoring Large-Scale Cloud-Native Infrastructure Using One-Sided RDMA

Cloud platforms for remote monitoring system: a comparative case study