Abstract:Over the past decade, metamorphic testing has gained rapidly increasing attention from both academia and industry, particularly thanks to its high efficacy on revealing real-life software faults in a wide variety of application domains. On the basis of a set of metamorphic relations among multiple software inputs and their expected outputs, metamorphic testing not only provides a test case generation strategy by constructing new (or follow-up) test cases from some original (or source) test cases, but also a test result verification mechanism through checking the relationship between the outputs of source and follow-up test cases. Many efforts have been made to further improve the cost-effectiveness of metamorphic testing from different perspectives. Some studies attempted to identify “good” metamorphic relations, while other studies were focused on applying effective test case generation strategies especially for source test cases. In this article, we propose improving the cost-effectiveness of metamorphic testing by leveraging the feedback information obtained in the test execution process. Consequently, we develop a new approach, namely feedback-directed metamorphic testing, which makes use of test execution information to dynamically adjust the selection of metamorphic relations and selection of source test cases. We conduct an empirical study to evaluate the proposed approach based on four laboratory programs, one GNU program, and one industry program. The empirical results show that feedback-directed metamorphic testing can use fewer test cases and take less time than the traditional metamorphic testing for detecting the same number of faults. It is clearly demonstrated that the use of feedback information about test execution does help enhance the cost-effectiveness of metamorphic testing. Our work provides a new perspective to improve the efficacy and applicability of metamorphic testing as well as many other software testing techniques.

Metamorphic Relation Recognition Method Based on Control Flow Graph Features

Metamorphic Relations Recognition Method Based on Node Features and Path Features

Predicting Metamorphic Relations Based on Path Features

Metamorphic Relation Generation: State of the Art and Visions for Future Research

Metamorphic Relation Construction Method Based on Compositional Function

Towards Automated Metamorphic Test Identification for Ocean System Models

A Similarity-based Metamorphic Relations Selection Strategy for Numerical Computation Programs

Predicting Metamorphic Relation for Matrix Calculation Programs

Syntax‐based metamorphic relation prediction via the bagging framework

An Application of Metamorphic Testing for Testing Scientific Software.

Automated identification of metamorphic test scenarios for an ocean-modeling application

Feedback-Directed Metamorphic Testing

Tabular-expression-based Method for Constructing Metamorphic Relations.

Calculation Method of Metamorphic Relational Complexity for Numerical Computation Programs Based on Scale Complexity

Search-based Inference of Polynomial Metamorphic Relations

Identifying Metamorphic Relations: A Data Mutation Directed Approach

Identification Algorithm Framework and Structural Model on Input Pattern of Metamorphic Relations

Composite metamorphic relations for integration testing

Improve the Quality of ARC Systems Based on the Metamorphic Testing

Using Semi-Supervised Learning for Predicting Metamorphic Relations

Self-Checked Metamorphic Testing of an Image Processing Program