Fuzzy Clustering of Crowdsourced Test Reports for Apps.
He Jiang,Xin Chen,Tieke He,Zhenyu Chen,Xiaochen Li
DOI: https://doi.org/10.1145/3106164
2018-01-01
Abstract:DevOps is a new approach to drive a seamless Application (App) cycle from development to delivery. As a critical part to promote the successful implementation of DevOps, testing can significantly improve team productivity and reliably deliver user experience. However, it is difficult to use traditional testing to cover diverse mobile phones, network environments, operating systems, and so on. Hence, many large companies crowdsource their App testing tasks to workers from open platforms. In crowdsourced testing, test reports submitted by workers may be highly redundant, and their quality may vary sharply. Meanwhile, multi-bug test reports may be submitted, and their root causes are hard to diagnose. Hence, it is a time-consuming and tedious task for developers to manually inspect these test reports. To help developers address the above challenges, we issue the new problem of Fuzzy Clustering Test Reports (FULTER). Aiming to resolve FULTER, a series of barriers need to be overcome. In this study, we propose a new framework named Test Report Fuzzy Clustering Framework (TERFUR) by aggregating redundant and multi-bug test reports into clusters to reduce the number of inspected test reports. First, we construct a filter to remove invalid test reports to break through the invalid barrier. Then, a preprocessor is built to enhance the descriptions of short test reports to break through the uneven barrier. Last, a two-phase merging algorithm is proposed to partition redundant and multi-bug test reports into clusters that can break through the multi-bug barrier. Experimental results over 1,728 test reports from five industrial Apps show that TERFUR can cluster test reports by up to 78.15% in terms of AverageP, 78.41% in terms of AverageR, and 75.82% in terms of AverageF1 and outperform comparative methods by up to 31.69%, 33.06%, and 24.55%, respectively. In addition, the effectiveness of TERFUR is validated in prioritizing test reports for manual inspection.