Predicting Node Failures in an Ultra-Large-Scale Cloud Computing Platform
Yangguang Li,Zhen Ming Jiang,Heng Li,Ahmed E. Hassan,Cheng He,Ruirui Huang,Zhengda Zeng,Mian Wang,Pinan Chen,Zhen Ming (Jack) Jiang
DOI: https://doi.org/10.1145/3385187
IF: 3.685
2020-04-29
ACM Transactions on Software Engineering and Methodology
Abstract:Many software services today are hosted on cloud computing platforms, such as Amazon EC2, due to many benefits like reduced operational costs. However, node failures in these platforms can impact the availability of their hosted services and potentially lead to large financial losses. Predicting node failures before they actually occur is crucial, as it enables DevOps engineers to minimize their impact by performing preventative actions. However, such predictions are hard due to many challenges like the enormous size of the monitoring data and the complexity of the failure symptoms. AIOps ( A rtificial I ntelligence for IT Op eration s ), a recently introduced approach in DevOps, leverages data analytics and machine learning to improve the quality of computing platforms in a cost-effective manner. However, the successful adoption of such AIOps solutions requires much more than a top-performing machine learning model. Instead, AIOps solutions must be trustable, interpretable, maintainable, scalable, and evaluated in context. To cope with these challenges, in this article we report our process of building an AIOps solution for predicting node failures for an ultra-large-scale cloud computing platform at Alibaba. We expect our experiences to be of value to researchers and practitioners, who are interested in building and maintaining AIOps solutions for large-scale cloud computing platforms.
computer science, software engineering