Abstract:Combinatorial interaction testing (CIT) is an effective paradigm for testing highly configurable systems, and its goal is to generate a t-wise covering array (CA) as a test suite, where t is the strength of testing. It is recognized that pairwise testing (i.e., CIT with t=2) is the most common CIT technique, and has high fault detection capability in practice. The problem of pairwise CA generation (PCAG), which is a core problem in pairwise testing, aims at generating a pairwise CA (i.e., 2-wise CA) of minimum size, subject to hard constraints. The PCAG problem is a hard combinatorial optimization problem, which urgently requires practical methods for generating pairwise CAs (PCAs) of small sizes. However, existing PCAG algorithms suffer from the severe scalability issue; that is, when solving large-scale PCAG instances, existing state-of-the-art PCAG algorithms usually cost a fairly long time to generate large PCAs, which would make the testing of highly configurable systems both ineffective and inefficient. In this paper, we propose a novel and effective sampling-based approach dubbed SamplingCA for solving the PCAG problem. SamplingCA first utilizes sampling techniques to obtain a small test suite that covers valid pairwise tuples as many as possible, and then adds a few more test cases into the test suite to ensure that all valid pairwise tuples are covered. Extensive experiments on 125 public PCAG instances show that our approach can generate much smaller PCAs than its state-of-the-art competitors, indicating the effectiveness of SamplingCA. Also, our experiments show that SamplingCA runs one to two orders of magnitude faster than its competitors, demonstrating the efficiency of SamplingCA. Our results confirm that SamplingCA is able to address the scalability issue and considerably pushes forward the state of the art in PCAG solving.

A Memetic Algorithm for High-Strength Covering Array Generation.

Effective Discrete Memetic Algorithms for Covering Array Generation.

A tolerance-based memetic algorithm for constrained covering array generation

An Effective Approach to High Strength Covering Array Generation in Combinatorial Testing

Towards More Efficient Meta-Heuristic Algorithms for Combinatorial Test Generation.

FastCA: An Effective and Efficient Tool for Combinatorial Covering Array Generation

Generating Pairwise Covering Arrays for Highly Configurable Software Systems.

A Constrained Covering Array Generator Using Adaptive Penalty Based Parallel Tabu Search

<i>AutoCCAG</i>: An Automated Approach to Constrained Covering Array Generation

TCA: an Efficient Two-Mode Meta-Heuristic Algorithm for Combinatorial Test Generation (T).

A method of combinatorial test cases generating based on CMA-PGA model

A Hybrid Encoded Memetic Algorithm for Set Covering Problem

A Fast and Robust Heuristic Algorithm for the Minimum Weight Vertex Cover Problem

CAmpactor: A Novel and Effective Local Search Algorithm for Optimizing Pairwise Covering Arrays

An Adaptive Penalty Based Parallel Tabu Search for Constrained Covering Array Generation

A New Pairwise Covering Test Data Generation Algorithm for the System with Many 2-Level Factors

Extended Covering Arrays for Sequence Coverage.

Greedy Heuristic Algorithms to Generate Variable Strength Combinatorial Test Suite.

Search Based Combinatorial Testing

SamplingCA: Effective and Efficient Sampling-Based Pairwise Testing for Highly Configurable Software Systems.

Empirically Identifying the Best Greedy Algorithm for Covering Array Generation