Abstract:The test failure causes analysis is critical since it determines the subsequent way of handling different types of bugs, which is the prerequisite to get the bugs properly analyzed and fixed. After a test case fails, software testers have to inspect the test execution logs line by line to identify its root cause. However, manual root cause determination is often tedious and time-consuming, which can cost 30-40% of the time needed to fix a problem. Therefore, there is a need for automatically predicting the test failure causes to lighten the burden of software testers. In this paper, we present a simple but hard-to-beat approach, named NCChecker to automatically identify the failure causes for failed test logs. Our approach can help developers efficiently identify the test failure causes, and flag the most probable log lines of indicating the root causes for investigation. Our approach has three main stages: log abstraction, lookup table construction, and failure causes prediction. We first perform log abstraction to parse the unstructured log messages into structured log events. NCChecker then automatically maintains and updates a lookup table via employing our heuristic rules, which record the matching score between different log events and test failure causes. When it comes to the failure cause prediction stage, for a newly generated failed test log, NCChecker can easily infer its failed reason by checking out the associated log events' scores from the lookup table. We have developed a prototype and evaluated our tool on a real-world industrial dataset with more than 10K test logs. The extensive experiments show the promising performance of our model over a set of benchmarks. Moreover, our approach is highly efficient and memory-saving, and can successfully handle the data imbalance problem.

From Missteps to Milestones: A Journey to Practical Fail-Slow Detection

Perseus: A Fail-Slow Detection Framework for Cloud Storage Systems.

NVMe SSD Failures in the Field: the Fail-Stop and the Fail-Slow

A Failure Prediction Approach Based on BiLSTM and Deep Feature Extractor for Hard Disk Drives.

A Systematic Study of Failure Proximity

Understanding and taming SSD read performance variability: HDFS case study

General Feature Selection for Failure Prediction in Large-scale SSD Deployment

Improving Productivity and Efficiency of SSD Manufacturing Self-Test Process by Learning-Based Proactive Defect Prediction

What Can We Learn from Four Years of Data Center Hardware Failures?

An In-Depth Study Of Correlated Failures In Production Ssd-Based Data Centers

Towards Learned Predictability of Storage Systems

When Amnesia Strikes: Understanding and Reproducing Data Loss Bugs with Fault Injection

Hard Drive Failure Prediction Using Big Data

Detection Software Content Failures Using Dynamic Execution Information

New Insights Into Defect Loss, Slowdown, and Device Lifetime Enhancement

Easy over Hard: A Simple Baseline for Test Failures Causes Prediction

Mitigating Synchronous I/O Overhead in File Systems on Open-Channel SSDs

Toward Adaptive Disk Failure Prediction Via Stream Mining

ADF2T: an Active Disk Failure Forecasting and Tolerance Software

Hard Disk Drives Failure Detection Using A Dynamic Tracking Method

Large Scale Studies of Memory, Storage, and Network Failures in a Modern Data Center