A Model-Based Approach for Specifying Changes in Replications of Empirical Studies in Computer Science

Margarita Cruz,Beatriz Bernárdez,Amador Durán,Cathy Guevara-Vega,Antonio Ruiz-Cortés
2022-06-27
Abstract:Context: The need of replicating empirical studies in Computer Science (CS) is widely recognized among the research community to consolidate acquired knowledge generalizing results. It is essential to report the changes of each replication to understand the evolution of the experimental validity across a family of studies. Unfortunately, the lack of proposals undermines these objectives.
Computer Science
What problem does this paper attempt to address?
The problem that this paper attempts to solve is the lack of systematic reporting and specification changes in the replication of empirical research in computer science and other research fields. Specifically, the paper points out that although replicating experiments is crucial for consolidating the results of previous research and generalizing the results, there is currently a lack of effective tools and methods to systematically record and report the changes in these replication experiments. This deficiency makes it difficult for new replicators to decide which aspects of the experimental setup are most suitable for the new environment and to avoid the mistakes in previous experiments, thus affecting the validity and repeatability of the experiments. Therefore, the main objective of the paper is to provide a systematic and tool - supported method for specifying and reporting changes in empirical research replication to improve the transparency and repeatability of research.