Abstract:The capability of Quantum Computing (QC) in solving complex problems has been increasingly recognized. However, similar to classical computing, to fully exploit QC's potential, it is important to ensure the correctness of quantum programs. Doing so via software testing is, however, very challenging because of QC's inherent properties: superposition and entanglement. Towards the direction of ensuring the correctness of quantum programs, we propose an approach called QuCAT (QUantum CombinAtorial Testing) for systematic and automated testing of quantum programs by benefiting from combinatorial testing, which has been proven to be cost-effective in testing classical programs. QuCAT supports two combinatorial test suite generation scenarios, i.e., generating combinatorial test suites of a given strength, and incrementally generating and executing combinatorial test suites of increasing strength until a fault is found. The approach employs two types of test oracles to assess test results. We performed an empirical study with 18 faulty versions of quantum programs to evaluate QuCAT with strengths of two, three, and four in the two test generation scenarios. We compare the cost-effectiveness of combinatorial testing of various strengths and random testing (taken as baseline approach). Results show that combinatorial testing always performs better than random testing with the same cost and finds faults more quickly (in terms of required number of test cases). In addition, in most cases, combinatorial testing with a higher strength outperforms the lower strength in terms of effectiveness.

Optimal Test Suite Generation Methods for Neighbor Factors Combinatorial Testing

Generating Optimal Test Set for Neighbor Factors Combinatorial Testing

Generating combinatorial test suite for interaction relationship.

Greedy Heuristic Algorithms to Generate Variable Strength Combinatorial Test Suite.

Survey of combinatorial test generation

Generating Variable Strength Combinatorial Test Suite with One-test-at-a-time Strategy

A New Pairwise Covering Test Data Generation Algorithm for the System with Many 2-Level Factors

Towards Optimal Concolic Testing

Automatic Test Generation for N-Way Combinatorial Testing

Combinatorial Design Approaches for Automatic Test Generation

An Effective Approach to High Strength Covering Array Generation in Combinatorial Testing

Case Generation by Constraints Combinatorial Testing

PRIORITIZATION OF COMBINATORIAL TEST CASES BY INCREMENTAL INTERACTION COVERAGE

Application of Combinatorial Testing to Quantum Programs

Software fault diagnosis method based on combinatorial testing

A Case Study of Adaptive Combinatorial Testing

Adaptive Interaction Fault Location Based on Combinatorial Testing

Adaptive Combinatorial Testing

An Interleaving Approach to Combinatorial Testing and Failure-Inducing Interaction Identification

A survey of combinatorial testing

Generate Test Cases for Combinatorial Testing Based on Genetic Algorithm