Abstract:Summary Continuous integration, at its core, includes a set of practices that aim to prevent and reduce the cost of software integration issues by merging working software copies often. Regression testing is considered a good practice in software development with continuous integration, which ensures that code changes are not negatively affecting software functionality. As, nowadays, software development is carried out iteratively, with small code increments continuously developed and regression tested, it is of critical importance that continuous regression testing is time efficient. However, in practice, regression testing is often long lasting and faces scalability problems as software grows larger or as software changes are made more frequently. One contributing factor to these issues is test redundancy, which causes the same software functionality being tested multiple times across a test suite. In large‐scale software, especially highly configurable software, redundancy in continuous regression testing can significantly grow the size of test suites and negatively affect the cost effectiveness of continuous integration. This paper presents a practical learning algorithm for optimizing continuous integration testing by reducing ineffective test redundancy in regression suites. The novelty of the algorithm lies in learning and predicting the fault‐detection effectiveness of continuous integration tests using historical test records and combining this information with coverage‐based redundancy metrics. The goal is to identify ineffective redundancy, which is maximally reduced in the resulting regression test suite, thus reducing test time and improving the performance of continuous integration. We apply and evaluate the algorithm in two industrial projects of continuous integration. The results show that the proposed algorithm can improve the efficiency of continuous integration practice in terms of decreasing test execution time by 38% on average compared to the industry practice of our case study and by 40% on average compared to the retest‐all approach. The results further demonstrate no significant reduction in fault‐detection effectiveness of continuous regression testing. This suggests that the proposed algorithm contributes to the state of the practice in the continuous integration development and testing of highly configurable systems.

Efficient Summary Reuse for Software Regression Verification

When Regression Verification Meets CEGAR.

Parallel Iterative Reengineering Model of Legacy Systems.

Applying Rely-Guarantee Reasoning on Concurrent Memory Management and Mailbox in C/OS-II: A Case Study

Incremental Predicate Analysis for Regression Verification.

Learning-Based Assume-Guarantee Regression Verification

Combining Model Checking and Testing with an Application to Reliability Prediction and Distribution

Regression Test Cases Generation Based on Automatic Model Revision

SRRTA: Regression Testing Acceleration Via State Reuse.

Automatic Approach of Generating Summaries for Common Loops and Its Application

Scalable Isolation of Failure-Inducing Changes Via Version Comparison.

Study of Regression Testing Tool Based on Subversion

Scalable Verification Framework for C Program

Cga: Combining Cluster Analysis With Genetic Algorithm For Regression Suite Reduction Of Microprocessors

Improving Simulation Regression Efficiency using a Machine Learning-based Method in Design Verification

Termination Analysis for Evolving Programs: an Incremental Approach by Reusing Certified Modules

A learning algorithm for optimizing continuous integration development and testing practice

DiffGen: Automated Regression Unit-Test Generation

RegMiner: Towards Constructing a Large Regression Dataset from Code Evolution History

Automated Behavioral Regression Testing

Software design for reliability and reuse: a proof-of-concept demonstration