Abstract:This issue contains four papers. The first paper provides a survey of work on testing adaptive and context-aware systems, while the second one concerns testing embedded systems. The remaining two papers explore particular problems associated with an area well known to most STVR readers: mutation testing. The first paper, ‘Testing of adaptive and context-aware systems: Approaches and challenges’, by Bento R. Siqueira, Fabiano C. Ferrari, Kathiani E. Souza, Valter V. Camargo and Rogério de Lemos, introduces a systematic literature review and a thematic analysis of studies to characterize the state of the art in testing adaptive systems (ASs) and context-aware systems (CASs) and discuss approaches, challenges, observed trends and research limitations and directions. The authors discover recurring research concerns related to AS and CAS testing (such as generation of test cases and built-in tests), recurring testing challenges (such as context monitoring and runtime decisions), some trends (such as model-based testing and hybrid techniques) and some little investigated issues (such as uncertainty and prediction of changes). (Recommended by T.Y. Chen) The second paper, ‘Remote embedded devices test framework on the cloud’, by Il-Seok (Benjamin) Choi and Chang-Sung Jeong, introduces a remote embedded device test framework on the cloud named RED-TFC, whose reliability test manager component can automatically perform various tests for evaluating reliability and performance of distributed shared devices by utilizing the cloud concept. RED-TFC includes two major techniques: the adaptive sample scale for reliability test (ASRT) and the mass sample reliability test (MSRT). The authors analyse two Android smartphone models that include many embedded components and show that RED-TFC can help detect a high number of reliability problems in smartphones. (Recommended by Tanja Vos) The third paper, ‘Analysing the combination of cost reduction techniques in Android mutation testing’, by Macario Polo-Usaola and Isyed Rodríguez-Trujillo, concerns the use of mutation testing when testing mobile apps. As the authors note, when testing an app, one typically deploys the app and its mutants on mobile devices or executes them on an emulator. Doing so increases the test execution time. Naturally, it can also significantly increase the cost of mutation testing, especially when there are many mutants. The authors investigate several techniques that have been devised for reducing execution time in mutation testing and produce a mathematical model with the aim of predicting the time taken when some combination of these techniques is used. (Recommended by Mike Papadakis) The final paper is ‘An ensemble-based predictive mutation testing approach that considers impact of unreached mutants’ by Alireza Aghamohammadi and Seyed-Hassan Mirian-Hosseinabadi. This paper also concerns both mutation testing and prediction. However, the authors look at a different prediction problem: that of predicting whether a mutant will be killed. The authors note that previous work did not consider the impact of unreachable mutants: those where the mutation point is not covered by any of the test cases used. It is argued that since many mutation tools exclude unreachable mutants, such mutants should also be removed from any empirical evaluation. The authors report the results of replicating previous studies but also eliminating unreachable mutants, finding that the resultant performance of prediction techniques is far lower than that reported. The authors then propose an alternative prediction model, which is shown to be effective when unreachable mutants are removed. (Recommended by Tanja Vos)

Regression mutation testing.

Predictive Mutation Testing

Mutation Testing in Evolving Systems: Studying the relevance of mutants to code evolution

Mutation Testing: Principal, Optimization and Application

MMT: Mutation Testing of Java Bytecode with Model Transformation -- An Illustrative Demonstration

Testing multiple linear regression systems with metamorphic testing

MQP: Mutants Quality Prediction for Cost-Effective Mutation Testing

Machine Learning and Evolutionary Computing for GUI-based Regression Testing

Binary Mutation Testing Through Dynamic Translation

Test Data Generation for Mutation Testing Based on Markov Chain Usage Model and Estimation of Distribution Algorithm

An Empirical Study on the Scalability of Selective Mutation Testing

Mutation Testing framework for Machine Learning

Improving Defect Detection Ability of Derived Test Cases Based on Mutated UML Activity Diagrams

What Are We Really Testing in Mutation Testing for Machine Learning? A Critical Reflection

A Cost-effective and Machine-learning-based method to identify and cluster redundant mutants in software mutation testing

Does mutation testing improve testing practices?

Efficient Mutation Testing via Pre-Trained Language Models

Application of Metamorphic Testing Monitored by Test Adequacy in a Monte Carlo Simulation Program.

Mutation-driven Test Case Generation Using Short-lived Concurrent Mutants -- First Results

Towards Mutation Testing of Reinforcement Learning Systems

Adaptive or Embedded Software Testing and Mutation Testing