An Empirical Study on Constraint Optimization Techniques for Test Generation
Zhiyi Zhang,Zhenyu Chen,Ruizhi Gao,Eric Wong,Baowen Xu
DOI: https://doi.org/10.1007/s11432-015-0450-5
2016-01-01
Science China Information Sciences
Abstract:Constraint solving is a frequent, but expensive operation with symbolic execution to generate tests for a program. To improve the efficiency of test generation using constraint solving, four optimization techniques are usually applied to existing constraint solvers, which are constraint independence, constraint set simplification, constraint caching, and expression rewriting. In this paper, we conducted an empirical study, using these four constraint optimization techniques in a well known test generation tool KLEE with 77 GNU Coreutils applications, to systematically investigate how these optimization techniques affect the efficiency of test generation. The experimental results show that these constraint optimization techniques as well as their combinations cannot improve the efficiency of test generation significantly for ALL-SIZED programs. Moreover, we studied the constraint optimization techniques with respect to two static metrics, lines of code(LOC) and cyclomatic complexity(CC), of programs. The experimental results show that the "constraint set simplification" technique can improve the efficiency of test generation significantly for the programs with high LOC and CC values. The"constraint caching" optimization technique can improve the efficiency of test generation significantly for the programs with low LOC and CC values. Finally, we propose four hybrid optimization strategies and practical guidelines based on different static metrics.