Abstract:Summary Continuous integration, at its core, includes a set of practices that aim to prevent and reduce the cost of software integration issues by merging working software copies often. Regression testing is considered a good practice in software development with continuous integration, which ensures that code changes are not negatively affecting software functionality. As, nowadays, software development is carried out iteratively, with small code increments continuously developed and regression tested, it is of critical importance that continuous regression testing is time efficient. However, in practice, regression testing is often long lasting and faces scalability problems as software grows larger or as software changes are made more frequently. One contributing factor to these issues is test redundancy, which causes the same software functionality being tested multiple times across a test suite. In large‐scale software, especially highly configurable software, redundancy in continuous regression testing can significantly grow the size of test suites and negatively affect the cost effectiveness of continuous integration. This paper presents a practical learning algorithm for optimizing continuous integration testing by reducing ineffective test redundancy in regression suites. The novelty of the algorithm lies in learning and predicting the fault‐detection effectiveness of continuous integration tests using historical test records and combining this information with coverage‐based redundancy metrics. The goal is to identify ineffective redundancy, which is maximally reduced in the resulting regression test suite, thus reducing test time and improving the performance of continuous integration. We apply and evaluate the algorithm in two industrial projects of continuous integration. The results show that the proposed algorithm can improve the efficiency of continuous integration practice in terms of decreasing test execution time by 38% on average compared to the industry practice of our case study and by 40% on average compared to the retest‐all approach. The results further demonstrate no significant reduction in fault‐detection effectiveness of continuous regression testing. This suggests that the proposed algorithm contributes to the state of the practice in the continuous integration development and testing of highly configurable systems.

Introducing Automated Regression Testing in Open Source Projects

Comparative Case Studies of Open Source Software Peer Review Practices

Open innovation using open source tools: a case study at Sony Mobile

Risk Based Testing of Open Source Software (OSS)

Automated Behavioral Regression Testing

A Large-Scale Evaluation of Automated Unit Test Generation Using EvoSuite

Promoting open science in test-driven software experiments

An Exploratory Study of the State of Practice of Performance Testing in Java-Based Open Source Projects

An Automated System for Interactively Learning Software Testing

Onboarding in Open Source Software Projects: A Preliminary Analysis

A learning algorithm for optimizing continuous integration development and testing practice

Data Driven Testing of Open Source Software.

No Free Lunch: Research Software Testing in Teaching

How Developers Engineer Test Cases: An Observational Study

Parameterized Unit Testing in the Open Source Wild

A Roadmap for Software Testing in Open Collaborative Development Environments

An Experience Report On Applying Software Testing Academic Results In Industry: We Need Usable Automated Test Generation

Building an open-source system test generation tool: lessons learned and empirical analyses with EvoMaster

Introducing High School Students to Version Control, Continuous Integration, and Quality Assurance

On Extracting Unit Tests from Interactive Programming Sessions

Time-Space Efficient Regression Testing for Configurable Systems