Integrated Test Case Generation and Prioritization Based on Multiple Coverage Criteria

Zhonghao Guo,Chenge Geng,Xinyue Xu,Xiangxian Chen

DOI: https://doi.org/10.21203/rs.3.rs-5307326/v1

2024-01-01

Abstract:Purpose: Software testing is crucial for ensuring software quality, but it often consumes significant time and resources. Test case generation tools can generate test cases which cover the elements of tested program. However, some types of test case generation algorithms, such as those based on symbolic execution, can also gather coverage information during the test case generation process, which is undoubtedly valuable for other testing tasks. Test case prioritization (TCP) helps detect software faults earlier, achieving better testing outcomes under limited resources. Our goal is to enhance TCP without increasing costs, making it applicable across all white-box testing while improving its effect. Methods: We developed MCTCG, a symbolic execution-based test case generation tool that produces test case sets meeting various coverage criteria. During generation, MCTCG analyzes program coverage for elements like statements and branches to guide the next step. We also proposed TCPall−first , a prioritization method that uses the coverage information of MCTCG to prioritize test cases based on the principle of 'maximizing coverage step by step.' It selects test cases that cover the most uncovered elements until full coverage is achieved. Since TCPall−first directly uses the coverage data of MCTCG, there is no need to re-execute the test cases to determine priority, allowing test case generation and prioritization to occur in the same process, saving significant testing resources. Results and conclusion: We evaluated our approach using multiple open-source programs. The results show that MCTCG successfully generates high-coverage test case sets according to various coverage criterias, with higher bug detection rates. TCPall-first uses the coverage information obtained during the generation process to determine test case prioritization. Compared with the best performing among other TCP methods, TCPall−first improved the APFD value by 4.11% and reduced the first-fail metric by 8.7%. Moreover, TCPall−first required the fewest test cases to detect the same number of bugs compared to all other methods. Therefore, we believe that MCTCG effectively enhances test case set coverage while gathering coverage information, and TCPall−first extends the applicability of TCP and improves coverage-based TCP techniques.