Top-down: A better strategy for incremental covering array generation
Yan Wang,Xintao Niu,Huayao Wu,Changhai Nie,Lei Yu,Xiaoyin Wang,Jiaxi Xu
DOI: https://doi.org/10.1016/j.infsof.2024.107601
IF: 3.9
2024-11-01
Information and Software Technology
Abstract:Context: The Incremental Covering Array (ICA) offers a flexible and efficient test schedule for Combinatorial Testing (CT) by enabling dynamic adjustment of test strength. Despite its importance, ICA generation has been under-explored in the CT community, resulting in limited and suboptimal existing approaches. Objective: To address this gap, we introduce a novel strategy, namely Top-down , for optimizing ICA generation. Method: In contrast to the traditional strategy, named Bottom-up , Top-down starts with a higher-strength test set and then extracts lower-strength sets from it, thus leveraging test case generation algorithms more effectively. Results: We conducted a comparative evaluation of the two strategies across 17 real-world software with 84 total versions. The results demonstrate that compared with Bottom-up , the Top-down strategy requires less time and generates smaller ICAs while covering more higher-strength interactions. Furthermore, Top-down outperforms Bottom-up in early fault detection and code line coverage, while also surpassing the random and direct CA generation strategies. Conclusion: The Top-down strategy not only improved the efficiency of test case generation but also enhanced the effectiveness of fault detection in the incremental testing scenarios.
computer science, information systems, software engineering