Abstract:DevOps is a new approach to drive a seamless Application (App) cycle from development to delivery. As a critical part to promote the successful implementation of DevOps, testing can significantly improve team productivity and reliably deliver user experience. However, it is difficult to use traditional testing to cover diverse mobile phones, network environments, operating systems, and so on. Hence, many large companies crowdsource their App testing tasks to workers from open platforms. In crowdsourced testing, test reports submitted by workers may be highly redundant, and their quality may vary sharply. Meanwhile, multi-bug test reports may be submitted, and their root causes are hard to diagnose. Hence, it is a time-consuming and tedious task for developers to manually inspect these test reports. To help developers address the above challenges, we issue the new problem of Fuzzy Clustering Test Reports (FULTER). Aiming to resolve FULTER, a series of barriers need to be overcome. In this study, we propose a new framework named Test Report Fuzzy Clustering Framework (TERFUR) by aggregating redundant and multi-bug test reports into clusters to reduce the number of inspected test reports. First, we construct a filter to remove invalid test reports to break through the invalid barrier. Then, a preprocessor is built to enhance the descriptions of short test reports to break through the uneven barrier. Last, a two-phase merging algorithm is proposed to partition redundant and multi-bug test reports into clusters that can break through the multi-bug barrier. Experimental results over 1,728 test reports from five industrial Apps show that TERFUR can cluster test reports by up to 78.15% in terms of AverageP, 78.41% in terms of AverageR, and 75.82% in terms of AverageF1 and outperform comparative methods by up to 31.69%, 33.06%, and 24.55%, respectively. In addition, the effectiveness of TERFUR is validated in prioritizing test reports for manual inspection.

Clustering Study of Crowdsourced Test Report with Multi-source Heterogeneous Information.

Clustering Crowdsourced Test Reports of Mobile Applications Using Image Understanding

Semi-supervised Crowdsourced Test Report Clustering Via Screenshot-Text Binding Rules

SemCluster: a Semi-Supervised Clustering Tool for Crowdsourced Test Reports with Deep Image Understanding.

Classifying Crowdsourced Mobile Test Reports with Image Features: an Empirical Study

Fuzzy Clustering of Crowdsourced Test Reports for Apps.

CTRAS: Crowdsourced Test Report Aggregation and Summarization.

Clustering on the Stream of Crowdsourced Testing

STIFA: Crowdsourced Mobile Testing Report Selection Based on Text and Image Fusion Analysis

Multi-objective Test Report Prioritization Using Image Understanding

CTRAS: a tool for aggregating and summarizing crowdsourced test reports

Automated Quality Assessment for Crowdsourced Test Reports Based on Dependency Parsing

Mobile crowdsourced test report prioritization based on text and image understanding

Prioritize Crowdsourced Test Reports via Deep Screenshot Understanding

Optimizing Prioritization of Crowdsourced Test Reports of Web Applications through Image-to-Text Conversion

A Systemic Framework for Crowdsourced Test Report Quality Assessment

Summarizing the Crowdsourced Testing

Automatic test report augmentation to assist crowdsourced testing

Leveraging Android Automated Testing to Assist Crowdsourced Testing

Redefining Crowdsourced Test Report Prioritization: An Innovative Approach with Large Language Model

Guiding the Crowds for Android Testing