Abstract:This editorial was written during a period of extreme difficulty for many individuals, families, and nations in the ongoing COVID-19 outbreak. We can only hope that measures taken are successful and that the situation has improved considerably. We also do not pretend that we have anything to add regarding health, social, or economic issues. However, the crisis has shown the role that Computer Science can play in informing policy. Society requires evidence and computers are often involved in producing such evidence via, for example, simulation. It is here that we, as a community, can contribute through advances in testing, verification, and reliability in areas such as Scientific Computing and Computer Simulations and maybe also AI/data sciences for helping expedite the process of finding treatment. As a recent example discussed in social media, when commenting on pandemic simulation code used to model control measures against COVID-19, Prof. Guido Salvaneschi said in his tweet: “Ever wondered about the “impact“ of research on programming languages and software engineering? Political decisions affecting hundreds of millions are being taken based on thousands of lines of 13+ years old C code that allegedly nobody understands anymore. #COVID19 #cs” (https://twitter.com/guidosalva/status/1242049884347412482). There is already some truly excellent work for making advances in these areas and we are confident that the community will rise to the challenge. This issue contains two papers. In the first paper, Simons and Lefticaru introduce a new Model-Based Testing approach, which is based on the use of a Stream X-machine (SXM) specification. SXMs provide a state-based formalism and there is a traditional approach to testing from an SXM. This approach typically assumes that the underlying functions/operations have been implemented correctly but these functions may be integrated (into a state machine) in the wrong way. There are a number of automated test generation approaches for SXMs and the authors make two main additional contributions to this area. First, they introduce a number of novel optimisations into test generation. Second, they observe that SXM test generation algorithms return abstract test cases (sequences of functions); the paper shows how corresponding concrete test data can be generated. The approach has been implemented and evaluated on case studies, with the tool also checking that a specification satisfies certain desirable properties. (Recommended by Hyunsook Do). In the second paper, Pouria Derakhshanfar, Xavier Devroey, Gilles Perrouin, Andy Zaidman, and Arie van Deursen introduce behavioural model seeding, a new seeding approach for learning class usages from both the system source code under test and existing test cases. The learned class usages are represented in a state-machine-based behavioural model. The behavioural model is then used to guide search-based crash reproduction, which generates a test case (i.e., objects and sequences of method calls on those objects) to reproduce a crash given its stack trace. This approach is in contrast to the existing seeding strategies, which simply collect and reuse values and object states from the system source code under test and existing test cases without any abstraction. The approach has been implemented in an open-source implementation named the BOTSING toolset and evaluated on 122 crashes from six open-source applications. (Recommended by Phil McMinn).

Promoting open science in test-driven software experiments

LiveDocs: Crafting Interactive Development Environments From Research Findings

Open Science in Software Engineering

Working Across Boundaries.

Exploring Freely Available Data Tools to Support Open Data and Open Science

BEAT: An Open-Source Web-Based Open-Science Platform

"Share and Enjoy": Publishing Useful and Usable Scientific Models

Will it run?—A proof of concept for smoke testing decentralized data analytics experiments

Test-driven Software Experimentation with LASSO: an LLM Benchmarking Example

Sustainable Software Ecosystems for Open Science

Seamless Science: Lifting Experimental Mechanical Testing Lab Data to an Interoperable Semantic Representation

Provide Proactive Reproducible Analysis Transparency with Every Publication

MLDev: Data Science Experiment Automation and Reproducibility Software

Promoting an open research culture

Reproducibility in Research: Systems, Infrastructure, Culture

Introduction to Special Section on Search Based Software Engineering

Open Reproducible Publication Research

Toward Reusable Science with Readable Code and Reproducibility

Computing Environments for Reproducibility: Capturing the "Whole Tale"

SciOps: Achieving Productivity and Reliability in Data-Intensive Research

KheOps: Cost-effective Repeatability, Reproducibility, and Replicability of Edge-to-Cloud Experiments