Abstract:This issue contains four papers. The first paper provides a survey of work on testing adaptive and context-aware systems, while the second one concerns testing embedded systems. The remaining two papers explore particular problems associated with an area well known to most STVR readers: mutation testing. The first paper, ‘Testing of adaptive and context-aware systems: Approaches and challenges’, by Bento R. Siqueira, Fabiano C. Ferrari, Kathiani E. Souza, Valter V. Camargo and Rogério de Lemos, introduces a systematic literature review and a thematic analysis of studies to characterize the state of the art in testing adaptive systems (ASs) and context-aware systems (CASs) and discuss approaches, challenges, observed trends and research limitations and directions. The authors discover recurring research concerns related to AS and CAS testing (such as generation of test cases and built-in tests), recurring testing challenges (such as context monitoring and runtime decisions), some trends (such as model-based testing and hybrid techniques) and some little investigated issues (such as uncertainty and prediction of changes). (Recommended by T.Y. Chen) The second paper, ‘Remote embedded devices test framework on the cloud’, by Il-Seok (Benjamin) Choi and Chang-Sung Jeong, introduces a remote embedded device test framework on the cloud named RED-TFC, whose reliability test manager component can automatically perform various tests for evaluating reliability and performance of distributed shared devices by utilizing the cloud concept. RED-TFC includes two major techniques: the adaptive sample scale for reliability test (ASRT) and the mass sample reliability test (MSRT). The authors analyse two Android smartphone models that include many embedded components and show that RED-TFC can help detect a high number of reliability problems in smartphones. (Recommended by Tanja Vos) The third paper, ‘Analysing the combination of cost reduction techniques in Android mutation testing’, by Macario Polo-Usaola and Isyed Rodríguez-Trujillo, concerns the use of mutation testing when testing mobile apps. As the authors note, when testing an app, one typically deploys the app and its mutants on mobile devices or executes them on an emulator. Doing so increases the test execution time. Naturally, it can also significantly increase the cost of mutation testing, especially when there are many mutants. The authors investigate several techniques that have been devised for reducing execution time in mutation testing and produce a mathematical model with the aim of predicting the time taken when some combination of these techniques is used. (Recommended by Mike Papadakis) The final paper is ‘An ensemble-based predictive mutation testing approach that considers impact of unreached mutants’ by Alireza Aghamohammadi and Seyed-Hassan Mirian-Hosseinabadi. This paper also concerns both mutation testing and prediction. However, the authors look at a different prediction problem: that of predicting whether a mutant will be killed. The authors note that previous work did not consider the impact of unreachable mutants: those where the mutation point is not covered by any of the test cases used. It is argued that since many mutation tools exclude unreachable mutants, such mutants should also be removed from any empirical evaluation. The authors report the results of replicating previous studies but also eliminating unreachable mutants, finding that the resultant performance of prediction techniques is far lower than that reported. The authors then propose an alternative prediction model, which is shown to be effective when unreachable mutants are removed. (Recommended by Tanja Vos)

Test Case Prioritization and Mutation Testing.

MUT: Human-in-the-Loop Unit Test Migration

Adaptive or Embedded Software Testing and Mutation Testing

Mutation Testing: Principal, Optimization and Application

Mutation-based test-case prioritization in software evolution

Integration Testing and Metamorphic Testing

Mutation Testing in Evolving Systems: Studying the relevance of mutants to code evolution

Metamorphic Testing and Test Automation

Heuristics for Strongly Killing Mutants in Software Mutation Testing

Test Case Generation using Mutation Operators and Fault Classification

Test Case Prioritization for Compilers: A Text-Vector Based Approach

Mutation Testing in Practice: Insights from Open-Source Software Developers

Does mutation testing improve testing practices?

In Vivo Testing and Integration of Proving and Testing

Predictive Mutation Testing

An Approach to Test Data Generation for Killing Multiple Mutants

Test-case Prioritization: Achievements and Challenges.

What Are We Really Testing in Mutation Testing for Machine Learning? A Critical Reflection

Test Data Generation for Mutation Testing Based on Markov Chain Usage Model and Estimation of Distribution Algorithm

Mutation-based Test Generation for Quantum Programs with Multi-Objective Search

MQP: Mutants Quality Prediction for Cost-Effective Mutation Testing