Abstract:<h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Context:</h3><p>Previous studies proposed different kinds of approaches for class integration test order generation, and corresponding systems can be implemented based on these approaches. Such class integration test order generation systems can facilitate the process of software integration testing if they are implemented correctly.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Objective:</h3><p>However, a test oracle problem exists in the class integration test order generation systems. Since these approaches for class integration test order generation normally deliver a local optimum rather than a global optimum, there are no practically feasible ways to validate their generated class integration test orders, that is, these implementation systems are untestable.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Method:</h3><p>To address the test oracle problem, we apply Metamorphic Testing (MT) to validate class integration test order generation systems. Metamorphic Relations (MRs), which are the key components of MT, reason about relations between test outputs of a system. Five effective MRs are developed to ensure the quality of the class integration test order generation systems. In these proposed MRs, follow-up test inputs are generated by modifying classes or class dependencies in the source test inputs while some characteristics of the source test outputs are preserved, for example, the same class integration test order or the equal stubbing cost. Faults can be detected in systems if an individual MR is violated for certain tests.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Results:</h3><p>Failure detection of MT has been successfully demonstrated in empirical experiments on three systems implementing different typical class integration test order generation approaches. More than 84% of faulty programs can be detected by all MRs, for three class integration test order generation systems under investigation.</p><h3 class="u-h4 u-margin-m-top u-margin-xs-bottom">Conclusion:</h3><p>The experimental results show that the proposed MRs are able to systematically and effectively detect faults in class integration test order generation systems. This study explores a new application domain in MT and further extends its applications in Software Engineering.</p>

Hierarchical Metamorphic Relations for Testing Scientific Software

Testing Scientific Software with Invariant Relations: A Case Study

An Application of Metamorphic Testing for Testing Scientific Software.

Application of Metamorphic Testing Monitored by Test Adequacy in a Monte Carlo Simulation Program.

Testing Ocean Software with Metamorphic Testing

Improving the Quality of Computational Science Software by Using Metamorphic Relations to Test Machine Learning Applications

Research Of Testing For Scientific Computing Software In The Area Of Nuclear Power Based On Metamorphic Testing

Tabular-expression-based Method for Constructing Metamorphic Relations.

Metamorphic Relation Generation: State of the Art and Visions for Future Research

Towards Automated Metamorphic Test Identification for Ocean System Models

Metamorphic Relations Identification on Chebyshev Rational Approximation Method in the Nuclide Depletion Calculation Program

Towards a Complete Metamorphic Testing Pipeline

An Iterative Metamorphic Testing Technique for Web Services and Case Studies.

Improve the Quality of ARC Systems Based on the Metamorphic Testing

Metamorphic relation prioritization for effective regression testing

Identifying Metamorphic Relations: A Data Mutation Directed Approach

Metamorphic Testing on Nuclide Inventory Tool

Validating class integration test order generation systems with Metamorphic Testing

Automated identification of metamorphic test scenarios for an ocean-modeling application

Search-based Inference of Polynomial Metamorphic Relations

METARO(3) : Metamorphic Relation Group for Automatic Program Repair