Abstract:Combinatorial interaction testing (CIT) is an effective paradigm for testing highly configurable systems, and its goal is to generate a t-wise covering array (CA) as a test suite, where t is the strength of testing. It is recognized that pairwise testing (i.e., CIT with t=2) is the most common CIT technique, and has high fault detection capability in practice. The problem of pairwise CA generation (PCAG), which is a core problem in pairwise testing, aims at generating a pairwise CA (i.e., 2-wise CA) of minimum size, subject to hard constraints. The PCAG problem is a hard combinatorial optimization problem, which urgently requires practical methods for generating pairwise CAs (PCAs) of small sizes. However, existing PCAG algorithms suffer from the severe scalability issue; that is, when solving large-scale PCAG instances, existing state-of-the-art PCAG algorithms usually cost a fairly long time to generate large PCAs, which would make the testing of highly configurable systems both ineffective and inefficient. In this paper, we propose a novel and effective sampling-based approach dubbed SamplingCA for solving the PCAG problem. SamplingCA first utilizes sampling techniques to obtain a small test suite that covers valid pairwise tuples as many as possible, and then adds a few more test cases into the test suite to ensure that all valid pairwise tuples are covered. Extensive experiments on 125 public PCAG instances show that our approach can generate much smaller PCAs than its state-of-the-art competitors, indicating the effectiveness of SamplingCA. Also, our experiments show that SamplingCA runs one to two orders of magnitude faster than its competitors, demonstrating the efficiency of SamplingCA. Our results confirm that SamplingCA is able to address the scalability issue and considerably pushes forward the state of the art in PCAG solving.

A Constrained Covering Array Generator Using Adaptive Penalty Based Parallel Tabu Search

An Adaptive Penalty Based Parallel Tabu Search for Constrained Covering Array Generation

<i>AutoCCAG</i>: An Automated Approach to Constrained Covering Array Generation

FastCA: An Effective and Efficient Tool for Combinatorial Covering Array Generation

Generating Pairwise Covering Arrays for Highly Configurable Software Systems.

Search Based Combinatorial Testing

TCA: an Efficient Two-Mode Meta-Heuristic Algorithm for Combinatorial Test Generation (T).

A tolerance-based memetic algorithm for constrained covering array generation

A Memetic Algorithm for High-Strength Covering Array Generation.

Effective Discrete Memetic Algorithms for Covering Array Generation.

CAmpactor: A Novel and Effective Local Search Algorithm for Optimizing Pairwise Covering Arrays

Towards More Efficient Meta-Heuristic Algorithms for Combinatorial Test Generation.

Extended Covering Arrays for Sequence Coverage.

An Effective Approach to High Strength Covering Array Generation in Combinatorial Testing

Comparative Analysis of Constraint Handling Techniques for Constrained Combinatorial Testing.

SamplingCA: Effective and Efficient Sampling-Based Pairwise Testing for Highly Configurable Software Systems.

Optimization of Configurable Greedy Algorithm for Covering Arrays Generation

A Combined Global And Local Search Method To Deal With Constrained Optimization For Continuous Tabu Search

Manifold-Inspired Search-Based Algorithm for Automated Test Case Generation

Refining a Randomized Post-optimization Method for Covering Arrays

Empirically Identifying the Best Greedy Algorithm for Covering Array Generation