Abstract:This issue contains four papers. The first paper provides a survey of work on testing adaptive and context-aware systems, while the second one concerns testing embedded systems. The remaining two papers explore particular problems associated with an area well known to most STVR readers: mutation testing. The first paper, ‘Testing of adaptive and context-aware systems: Approaches and challenges’, by Bento R. Siqueira, Fabiano C. Ferrari, Kathiani E. Souza, Valter V. Camargo and Rogério de Lemos, introduces a systematic literature review and a thematic analysis of studies to characterize the state of the art in testing adaptive systems (ASs) and context-aware systems (CASs) and discuss approaches, challenges, observed trends and research limitations and directions. The authors discover recurring research concerns related to AS and CAS testing (such as generation of test cases and built-in tests), recurring testing challenges (such as context monitoring and runtime decisions), some trends (such as model-based testing and hybrid techniques) and some little investigated issues (such as uncertainty and prediction of changes). (Recommended by T.Y. Chen) The second paper, ‘Remote embedded devices test framework on the cloud’, by Il-Seok (Benjamin) Choi and Chang-Sung Jeong, introduces a remote embedded device test framework on the cloud named RED-TFC, whose reliability test manager component can automatically perform various tests for evaluating reliability and performance of distributed shared devices by utilizing the cloud concept. RED-TFC includes two major techniques: the adaptive sample scale for reliability test (ASRT) and the mass sample reliability test (MSRT). The authors analyse two Android smartphone models that include many embedded components and show that RED-TFC can help detect a high number of reliability problems in smartphones. (Recommended by Tanja Vos) The third paper, ‘Analysing the combination of cost reduction techniques in Android mutation testing’, by Macario Polo-Usaola and Isyed Rodríguez-Trujillo, concerns the use of mutation testing when testing mobile apps. As the authors note, when testing an app, one typically deploys the app and its mutants on mobile devices or executes them on an emulator. Doing so increases the test execution time. Naturally, it can also significantly increase the cost of mutation testing, especially when there are many mutants. The authors investigate several techniques that have been devised for reducing execution time in mutation testing and produce a mathematical model with the aim of predicting the time taken when some combination of these techniques is used. (Recommended by Mike Papadakis) The final paper is ‘An ensemble-based predictive mutation testing approach that considers impact of unreached mutants’ by Alireza Aghamohammadi and Seyed-Hassan Mirian-Hosseinabadi. This paper also concerns both mutation testing and prediction. However, the authors look at a different prediction problem: that of predicting whether a mutant will be killed. The authors note that previous work did not consider the impact of unreachable mutants: those where the mutation point is not covered by any of the test cases used. It is argued that since many mutation tools exclude unreachable mutants, such mutants should also be removed from any empirical evaluation. The authors report the results of replicating previous studies but also eliminating unreachable mutants, finding that the resultant performance of prediction techniques is far lower than that reported. The authors then propose an alternative prediction model, which is shown to be effective when unreachable mutants are removed. (Recommended by Tanja Vos)

Fault Tolerance and Metamorphic Relation Prediction

Predictive Models in Software Engineering: Challenges and Opportunities

Unifying Defect Prediction, Categorization, and Repair by Multi-Task Deep Learning

Metamorphic Relation Generation: State of the Art and Visions for Future Research

Improving Early Fault Detection in Machine Learning Systems Using Data Diversity-Driven Metamorphic Relation Prioritization

Integration Testing and Metamorphic Testing

Syntax‐based metamorphic relation prediction via the bagging framework

Validating Unsupervised Machine Learning Techniques for Software Defect Prediction With Generic Metamorphic Testing

Property Generation/verification and Empirical Studies.

Investigating Fault Injection Techniques in Hardware-Based Deep Neural Networks and Mutation-Based Fault Localization

Using Semi-Supervised Learning for Predicting Metamorphic Relations

MUT Model: a metric for characterizing metamorphic relations diversity

Metamorphic Testing and Test Automation

Automatic Discovery and Cleansing of Numerical Metamorphic Relations

Improving the Quality of Computational Science Software by Using Metamorphic Relations to Test Machine Learning Applications

Adaptive or Embedded Software Testing and Mutation Testing

Metamorphic relation prioritization for effective regression testing

Coverage-enhanced fault diagnosis for Deep Learning programs: A learning-based approach with hybrid metrics

Security Versus Accuracy: Trade-Off Data Modeling to Safe Fault Classification Systems

MRGS‐ART: Metamorphic Relation and Group Selection Based on Adaptive Random Testing

Towards Fault Tolerance in Multi-Agent Reinforcement Learning