Abstract:Not all changes to a codebase are equal: Some modifications (e.g., heavy refactoring) are more critical than others (e.g., renaming local variables). In this paper, we present two regression test prioritization techniques, namely, method‐level tree kernel prioritization (MTK) and method‐level tree kernel with quotient set (MTK‐QS), leveraging tree kernel functions to effectively measure the structural similarity of changed methods and directing testing efforts towards code affected by more critical changes. Our experiments show that these techniques can significantly improve the fault detection rate than other traditional and widely used approaches. Regression test prioritization (RTP) is an active research field, aiming at re‐ordering the tests in a test suite to maximize the rate at which faults are detected. A number of RTP strategies have been proposed, leveraging different factors to reorder tests. Some techniques include an analysis of changed source code, to assign higher priority to tests stressing modified parts of the codebase. Still, most of these change‐based solutions focus on simple text‐level comparisons among versions. We believe that measuring source code changes in a more refined way, capable of discriminating between mere textual changes (e.g., renaming of a local variable) and more structural changes (e.g., changes in the control flow), could lead to significant benefits in RTP, under the assumption that major structural changes are also more likely to introduce faults. To this end, we propose two novel RTP techniques that leverage tree kernels (TK), a class of similarity functions largely used in Natural Language Processing on tree‐structured data. In particular, we apply TKs to abstract syntax trees of source code, to more precisely quantify the extent of structural changes in the source code, and prioritize tests accordingly. We assessed the effectiveness of the proposals by conducting an empirical study on five real‐world Java projects, also used in a number of RTP‐related papers. We automatically generated, for each considered pair of software versions (i.e., old version, new version) in the evolution of the involved projects, 100 variations with artificially injected faults, leading to over 5k different software evolution scenarios overall. We compared the proposed prioritization approaches against well‐known prioritization techniques, evaluating both their effectiveness and their execution times. Our findings show that leveraging more refined code change analysis techniques to quantify the extent of changes in source code can lead to relevant improvements in prioritization effectiveness, while typically introducing negligible overheads due to their execution.

Performance Regression Testing Target Prioritization Via Performance Risk Analysis

Distance Based Root Cause Analysis and Change Impact Analysis of Performance Regressions

PerfRanker: Prioritization of Performance Regression Tests for Collection-Intensive Software

How Does Regression Test Prioritization Perform in Real-World Software Evolution?

Optimizing test prioritization via test distribution analysis.

What Makes a Real Change in Software Performance? an Empirical Study on Analyzing the Factors That Affect the Triagement of Performance Change Points

Parallel Test Prioritization.

Locating Performance Regression Root Causes in the Field Operations of Web-based Systems: An Experience Report

Early Detection of Performance Regressions by Bridging Local Performance Data and Architectural Models

A Regression Test Case Prioritization Algorithm Based on Program Changes and Method Invocation Relationship

Test Report Prioritization To Assist Crowdsourced Testing

Regression test prioritization leveraging source code similarity with tree kernels

Using black-box performance models to detect performance regressions under varying workloads: an empirical study

Analyzing Performance-Aware Code Changes in Software Development Process

A Performance Testing Tool for Source Code

Evaluating Search-Based Software Microbenchmark Prioritization

FOREPOST: finding performance problems automatically with feedback-directed learning software testing

A History-Based Cost-Cognizant Test Case Prioritization Technique in Regression Testing

Failure risk management: adaptive performance control and mission abort decisions

Prioritizing JUnit test cases in absence of coverage information