Abstract:Cloud computing as a model to deliver distributed resource and services on the pay-as-you-go policy has become increasingly popular for all organizations including industry. However, the inherent dynamicity in this environment makes it prone to various types of performance problems which introduce many challenges in the area of distributed resource management. Advances in the big data learning approaches can bring the opportunity for a data aware dynamic management of resources in the cloud. The collected data from the performance indicators of the system can be a valuable source of information to identify unusual behaviors in the resource consumptions or application performance. Different types of problems can cause the performance degradations at VM or system level. System administrators are overwhelmed with the huge amount of data to be analyzed to find the problems and overall health of the system. In this paper, we argue that a better selection of dynamic resource scaling policies can be employed for better performance by predicting the anomalies in the system and narrowing down the possible cause of the anomaly to one of the attributes of the system. Therefore, we propose a 2-level cause aware auto-scaling framework which leverages two types of resource management solutions, horizontal and vertical, as the corrective actions when the performance is degraded. We show the effectiveness of vertical scaling strategy as a quick solution for cases that a VM is exposed to some type of the local anomaly, while the horizontal scaling solutions can be used for system wide anomaly to add new VMs in the system. Moreover, our data analysis module can predict anomalies to give sufficient time to the scaling system to make an effective scaling decision. The proposed unsupervised anomaly detection module leverages a new updating strategy for renewing the models which considers the changes in the state of the system to reduce the overhead of recurrent model trainings. We have performed a comparison of the proposed framework with an approach which is used by several popular cloud providers to show the advantage of mixing the multi-level auto-scaling with the knowledge of anomaly detection analysis in resolving performance problems in the cloud.

Performance anomaly detection using isolation‐trees in heterogeneous workloads of web applications in computing clouds

On the Effectiveness of Isolation‐based Anomaly Detection in Cloud Data Centers

ITL: an Isolation-Tree Based Learning of Features for Anomaly Detection in Networked Systems

ACAS: an Anomaly-Based Cause Aware Auto-Scaling Framework for Clouds

Parallel anomaly detection based on Isolation Forest

An Efficient Anomaly Detection Framework for Cloud Computing Environment

Data Anomaly Detection Based on Isolation Forest Algorithm

CloudDet: Interactive Visual Analysis of Anomalous Performances in Cloud Computing Systems

Isolation Forest Based Anomaly Detection Framework on Non-IID Data

Detecting Performance Anomalies in Scientific Workflows Using Hierarchical Temporal Memory

Performance-Aware Management of Cloud Resources: A Taxonomy and Future Directions

Anomaly Detection for Virtualized Data Center Via Outlier Analysis

Detecting Performance Anomaly with Correlation Analysis for Internetware

OptIForest: Optimal Isolation Forest for Anomaly Detection

AERF: Adaptive Ensemble Random Fuzzy Algorithm for Anomaly Detection in Cloud Computing.

WMS Distributed Resources Compute Node n Scientific Data Anomaly Detection Module Resource Monitor Compute Node 1 Resource Monitor Users QoS Requirements Resource Usage Data Stream

Utility Cloud: A Novel Approach for Diagnosis and Self-healing Based on the Uncertainty in Anomalous Metrics

Isolation-Based Anomaly Detection

LSHiForest: A Generic Framework for Fast Tree Isolation Based Ensemble Anomaly Analysis.

Intelligent Anomaly Detection and Mitigation in Data Centers