Abstract:The capability of Quantum Computing (QC) in solving complex problems has been increasingly recognized. However, similar to classical computing, to fully exploit QC's potential, it is important to ensure the correctness of quantum programs. Doing so via software testing is, however, very challenging because of QC's inherent properties: superposition and entanglement. Towards the direction of ensuring the correctness of quantum programs, we propose an approach called QuCAT (QUantum CombinAtorial Testing) for systematic and automated testing of quantum programs by benefiting from combinatorial testing, which has been proven to be cost-effective in testing classical programs. QuCAT supports two combinatorial test suite generation scenarios, i.e., generating combinatorial test suites of a given strength, and incrementally generating and executing combinatorial test suites of increasing strength until a fault is found. The approach employs two types of test oracles to assess test results. We performed an empirical study with 18 faulty versions of quantum programs to evaluate QuCAT with strengths of two, three, and four in the two test generation scenarios. We compare the cost-effectiveness of combinatorial testing of various strengths and random testing (taken as baseline approach). Results show that combinatorial testing always performs better than random testing with the same cost and finds faults more quickly (in terms of required number of test cases). In addition, in most cases, combinatorial testing with a higher strength outperforms the lower strength in terms of effectiveness.

Test Algebra for Combinatorial Testing

Adaptive Fault Detection for Testing Tenant Applications in Multi-tenancy SaaS Systems

Performing unit testing based on testing as a service (TaaS) approach

Isolating Failure-Inducing Combinations in Combinatorial Testing Using Test Augmentation and Classification

A survey of combinatorial testing

Testing As a Service over Cloud

Application of Combinatorial Testing to Quantum Programs

A Framework of Testing As a Service

Transition Algebra for Software Testing

Survey of combinatorial test generation

A Software Failure Debugging Method Based on Combinatorial Design Approach for Testing

An Algorithm for Fault Diagnosis Based on Combinatorial Design Approach for Testing

Testing and Fault Diagnosis for Web Application Compatibility Based on Combinatorial Method.

Adaptive Combinatorial Testing

Testing as a Service (TaaS) on Clouds

Combining Model Checking and Testing with an Application to Reliability Prediction and Distribution

Ontology-Based Test Modeling and Partition Testing of Web Services

Automatically Identifying Special and Common Unit Tests Based on Inferred Statistical Algebraic Abstractions

Software fault diagnosis method based on combinatorial testing

A Combinatorial Testing-Based Approach to Fault Localization

Generating combinatorial test suite for interaction relationship.