Actionable and Interpretable Fault Localization for Recurring Failures in Online Service Systems

Zeyan Li,Nengwen Zhao,Mingjie Li,Xianglin Lu,Lixin Wang,Dongdong Chang,Xiaohui Nie,Li Cao,Wenzhi Zhang,Kaixin Sui,Yanhua Wang,Xu Du,Guoqiang Duan,Dan Pei
DOI: https://doi.org/10.48550/arXiv.2207.09021
2022-07-19
Software Engineering
Abstract:Fault localization is challenging in an online service system due to its monitoring data's large volume and variety and complex dependencies across or within its components (e.g., services or databases). Furthermore, engineers require fault localization solutions to be actionable and interpretable, which existing research approaches cannot satisfy. Therefore, the common industry practice is that, for a specific online service system, its experienced engineers focus on localization for recurring failures based on the knowledge accumulated about the system and historical failures. Although the above common practice is actionable and interpretable, it is largely manual, thus slow and sometimes inaccurate. In this paper, we aim to automate this practice through machine learning. That is, we propose an actionable and interpretable fault localization approach, DejaVu, for recurring failures in online service systems. For a specific online service system, DejaVu takes historical failures and dependencies in the system as input and trains a localization model offline; for an incoming failure, the trained model online recommends where the failure occurs (i.e., the faulty components) and which kind of failure occurs (i.e., the indicative group of metrics) (thus actionable), which are further interpreted by both global and local interpretation methods (thus interpretable). Based on the evaluation on 601 failures from three production systems and one open-source benchmark, in less than one second, DejaVu can on average rank the ground truths at 1.66-th to 5.03-th among a long candidate list, outperforming baselines by at least 51.51%.
What problem does this paper attempt to address?