Abstract:Unit testing is critical to the software development process, ensuring the correctness of basic programming units in a program (e.g., a method). Search-based software testing (SBST) is an automated approach to generating test cases. SBST generates test cases with genetic algorithms by specifying the coverage criterion (e.g., branch coverage). However, a good test suite must have different properties, which cannot be captured using an individual coverage criterion. Therefore, the state-of-the-art approach combines multiple criteria to generate test cases. Since combining multiple coverage criteria brings multiple objectives for optimization, it hurts the test suites’ coverage for certain criteria compared with using the single criterion. To cope with this problem, we propose a novel approach named smart selection. Based on the coverage correlations among criteria and the subsumption relationships among coverage goals, smart selection selects a subset of coverage goals to reduce the number of optimization objectives and avoid missing any properties of all criteria. We conduct experiments to evaluate smart selection on 400 Java classes with three state-of-the-art genetic algorithms under the 2-minute budget. On average, smart selection outperforms combining all goals on 65.1% of the classes having significant differences between the two approaches. Secondly, we conduct experiments to verify our assumptions about coverage criteria relationships. Furthermore, we assess the coverage performance of smart selection under varying budgets of 5, 8, and 10 minutes and explore its effect on bug detection, confirming the advantage of smart selection over combining all goals.

Unit Selection: Case Study and Comparison with A/B Test Heuristic

Unit Selection with Nonbinary Treatment and Effect

Unit Selection: Learning Benefit Function from Finite Population Data

Unit Selection with Causal Diagram

An Algorithm and Complexity Results for Causal Unit Selection

A Unified Information Criterion For Evaluating Probe And Test Selection

Causal Unit Selection using Tractable Arithmetic Circuits

Invidious Comparisons: Ranking and Selection as Compound Decisions

Efficient Real-world Testing of Causal Decision Making via Bayesian Experimental Design for Contextual Optimisation

Tool-assisted Unit Test Selection Based on Operational Violations

Coverage Goal Selector for Combining Multiple Criteria in Search-Based Unit Test Generation

Novel item selection strategies for cognitive diagnostic computerized adaptive testing: A heuristic search framework

A Simple Way to Deal with Cherry-picking

An Analysis of Switchback Designs in Reinforcement Learning

Selectively Combining Multiple Coverage Goals in Search-Based Unit Test Generation

A quantitative approach to choose among multiple mutually exclusive decisions: comparative expected utility theory

Equivalence Test in Multi-dimensional Space with Applications in A/B Testing

Rapid and Scalable Bayesian AB Testing

Indifference-Zone-Free Selection of the Best

A Model and Heuristic For Solving Very Large Item Selection Problems

A simple model of decision-making in the application process