Abstract:One of the major problems in testing software code and assuring its quality is deriving test oracles. This problem becomes more evident when testing machine learning models. To overcome such problems, metamorphic testing is introduced. Metamorphic testing is built to test the presence of metamorphic relations; required relations that should hold for the program to be valid. Those metamorphic requirements are usually articulated by domain experts since they are application dependent which is a costly, complex, and error prone process. Therefore, the automatic detection of such relations can be more efficient for code verification. The novelty of this work is first emphasizing the two-way relationship between metamorphic testing and machine learning; where metamorphic testing is used to assess the effectiveness of machine learning models, while machine learning is used to automatically detect the main metamorphic relations in software code. Secondly, literature metamorphic relations are categorized into the eight main categories defined in literature, and three new metamorphic relations are introduced for the first time. Finally, all literature metamorphic relations are customized to fit the time series forecasting domain. An application is proposed consisting of a deep learning model for forecasting fresh produce yields along with a generalization framework to enable the proposed models to forecast other similar fresh produce yields. The interactive role played by metamorphic testing and machine learning is investigated through the quality assurance of the forecasting application. The datasets used to train and test the deep learning forecasting models as well as the forecasting models are verified using metamorphic tests and the metamorphic relations in the generalization code are automatically detected using support vector machine (SVM) models. Testing revealed the unmatched requirements that are fixed to have a valid application with sound data, effective models, and valid generalization code.

Improving the Quality of Computational Science Software by Using Metamorphic Relations to Test Machine Learning Applications

Application of Metamorphic Testing to Supervised Classifiers

Testing and Validating Machine Learning Classifiers by Metamorphic Testing.

An Application of Metamorphic Testing for Testing Scientific Software.

Improve the Quality of ARC Systems Based on the Metamorphic Testing

A Validation and Quality Assessment Method with Metamorphic Relations for Unsupervised Machine Learning Software.

Towards Effective Metamorphic Testing by Algorithm Stability for Linear Classification Programs

A study of the interactive role of metamorphic testing and machine learning in the quality assurance of a deep learning forecasting application

Research Of Testing For Scientific Computing Software In The Area Of Nuclear Power Based On Metamorphic Testing

Identifying implementation bugs in machine learning based image classifiers using metamorphic testing

Application of Metamorphic Testing Monitored by Test Adequacy in a Monte Carlo Simulation Program.

Testing Scientific Software with Invariant Relations: A Case Study

Feedback-Directed Metamorphic Testing

Hierarchical Metamorphic Relations for Testing Scientific Software

Quality Assurance of Bioinformatics Software: A Case Study of Testing a Biomedical Text Processing Tool Using Metamorphic Testing

METTLE: A Metamorphic Testing Approach To Validating Unsupervised Machine Learning Methods

Automated identification of metamorphic test scenarios for an ocean-modeling application

METTLE: A METamorphic Testing Approach to Assessing and Validating Unsupervised Machine Learning Systems

An Iterative Metamorphic Testing Technique for Web Services and Case Studies.

Testing multiple linear regression systems with metamorphic testing

Metamorphic Testing of Image Processing Applications: A General Framework and Optimization Strategies