Statistical debugging and automated program failure triage

Jiawei Han,Chao Liu

2008-01-01

Abstract:Recent years have seen great advances in software engineering and programming languages. Unfortunately, software is still far from bug-free, even for those deployed. Static analysis is a good approach to eliminating numerous bugs, but its conservative nature of analysis unavoidably constrains its capacity. Dynamic analysis, on the other hand, utilizes program runtime execution data, which complements static approaches. This thesis describes three dynamic techniques that leverage program runtime data to improve software quality.First, we present a statistical debugging algorithm, called S OBER, which automatically localizes software faults without any prior knowledge of the program semantics. This statistical debugging alleviates the high cost associated with manual debugging. Featuring a similar rationale to hypothesis testing, SOBER quantifies the fault relevance of each predicate in a principled way. We systematically evaluate S OBER under the same setting as previous studies, and compare S OBER with other seven algorithms. The result clearly demonstrates the advantages of SOBER over existing approaches.Second, we discuss automated program failure triage, which is a closely related problem with automated debugging. Recent software systems usually feature an automated failure reporting component, with which a huge number of failures are collected from software end-users. In order to effectively leverage the valuable program failure data, the collected failures need to be first triaged, i.e., to locate the most severe failures, and to assign them to appropriate developers. Lying in the center of failure triage is failure indexing, which tries to group failures due to the fault together. In this thesis, we propose a statistical debugging-based approach to program failure triage, called R-PROXIMITY, which better indexes failures and facilitates failure assignment than existing approaches. Finally, we describe a program dynamic slicing-based approach to failure indexing, which complements R-PROXIMITY. R-PROXIMITY is a quality failure indexing tool, but its effectiveness relies on a sufficient number of correct executions, which may or may not be available in practice. The proposed dynamic slicing-based approach does not require any correct executions, and hence perfectly complements R-PROXIMITY . All the three techniques are subject to three mid-sized programs grep, gzip, and flex, and the result validates their advancement of the state of the art.