Abstract:This editorial was written during a period of extreme difficulty for many individuals, families, and nations in the ongoing COVID-19 outbreak. We can only hope that measures taken are successful and that the situation has improved considerably. We also do not pretend that we have anything to add regarding health, social, or economic issues. However, the crisis has shown the role that Computer Science can play in informing policy. Society requires evidence and computers are often involved in producing such evidence via, for example, simulation. It is here that we, as a community, can contribute through advances in testing, verification, and reliability in areas such as Scientific Computing and Computer Simulations and maybe also AI/data sciences for helping expedite the process of finding treatment. As a recent example discussed in social media, when commenting on pandemic simulation code used to model control measures against COVID-19, Prof. Guido Salvaneschi said in his tweet: “Ever wondered about the “impact“ of research on programming languages and software engineering? Political decisions affecting hundreds of millions are being taken based on thousands of lines of 13+ years old C code that allegedly nobody understands anymore. #COVID19 #cs” (https://twitter.com/guidosalva/status/1242049884347412482). There is already some truly excellent work for making advances in these areas and we are confident that the community will rise to the challenge. This issue contains two papers. In the first paper, Simons and Lefticaru introduce a new Model-Based Testing approach, which is based on the use of a Stream X-machine (SXM) specification. SXMs provide a state-based formalism and there is a traditional approach to testing from an SXM. This approach typically assumes that the underlying functions/operations have been implemented correctly but these functions may be integrated (into a state machine) in the wrong way. There are a number of automated test generation approaches for SXMs and the authors make two main additional contributions to this area. First, they introduce a number of novel optimisations into test generation. Second, they observe that SXM test generation algorithms return abstract test cases (sequences of functions); the paper shows how corresponding concrete test data can be generated. The approach has been implemented and evaluated on case studies, with the tool also checking that a specification satisfies certain desirable properties. (Recommended by Hyunsook Do). In the second paper, Pouria Derakhshanfar, Xavier Devroey, Gilles Perrouin, Andy Zaidman, and Arie van Deursen introduce behavioural model seeding, a new seeding approach for learning class usages from both the system source code under test and existing test cases. The learned class usages are represented in a state-machine-based behavioural model. The behavioural model is then used to guide search-based crash reproduction, which generates a test case (i.e., objects and sequences of method calls on those objects) to reproduce a crash given its stack trace. This approach is in contrast to the existing seeding strategies, which simply collect and reuse values and object states from the system source code under test and existing test cases without any abstraction. The approach has been implemented in an open-source implementation named the BOTSING toolset and evaluated on 122 crashes from six open-source applications. (Recommended by Phil McMinn).

Crash reproduction via test case mutation: let existing test cases help.

MUT: Human-in-the-Loop Unit Test Migration

Automatic Solution Summarization for Crash Bugs

Improving Defect Detection Ability of Derived Test Cases Based on Mutated UML Activity Diagrams

Fixing Recurring Crash Bugs Via Analyzing Q&A Sites

Test Case Generation using Mutation Operators and Fault Classification

Improving Generation of Object-Oriented Test Suites by Avoiding Redundant Tests

Search-based Crash Reproduction using Behavioral Model Seeding

Mutation-driven Test Case Generation Using Short-lived Concurrent Mutants -- First Results

An Approach to Test Data Generation for Killing Multiple Mutants

Mutation-based test-case prioritization in software evolution

CrashJS: A NodeJS Benchmark for Automated Crash Reproduction

A New Mutation Analysis Method for Testing Java Exception Handling

Crash Consistency Validation Made Easy

Working Across Boundaries.

Mutation-Based Simulation Test Data Generation for Testing Complex Real-Time Software

A Fast Crash Reproduction Method for Android Applications Based On Widget Hierarchy Graphs

Test Case Prioritization and Mutation Testing.

Crashmaker

Does mutation testing improve testing practices?

Test Data Generation for Mutation Testing Based on Markov Chain Usage Model and Estimation of Distribution Algorithm