An assessment of heterogenous ensemble classifiers for analyzing change‐proneness in open‐source software systems

Megha Khanna,Ankita Bansal
DOI: https://doi.org/10.1002/smr.2660
2024-02-25
Journal of Software Evolution and Process
Abstract:Key Findings The results indicate the superiority of Average Probability Voting Ensemble, a heterogenous classifier for determining change‐proneness. The average AUC values developed using Average Probability Voting Ensemble exhibited an improvement of 3%‐9% and 3%‐11% respectively when compared with its base learners and homogeneous counter parts. Similar observations were inferred using other performance measures. The change in number of base learners or type of meta‐learner does not exhibit significant change in the performance of ensemble classifiers. Software managers constantly look out for methods that ensure cost effective development of good quality software products. An important means of accomplishing this is by allocating more resources to weak classes of a software product, which are prone to changes. Therefore, correct prediction of these change‐prone classes is critical. Though various researchers have investigated the performance of several algorithms for identifying them, the search for an optimum classifier still persists. To this end, this study critically investigates the use of six Heterogenous Ensemble Classifiers (HEC) for Software Change Prediction (SCP) by empirically validating datasets obtained from 12 open‐source software systems. The results of the study are statistically assessed using three robust performance indicators (AUC, F‐measure and Mathew Correlation Coefficient) in two different validation scenarios (within project and cross‐project). They indicate the superiority of Average Probability Voting Ensemble, a heterogenous classifier for determining change‐proneness in the investigated systems. The average AUC values of software change prediction models developed using this ensemble classifier exhibited an improvement of 3%‐9% and 3%‐11% respectively when compared with its base learners and homogeneous counter parts. Similar observations were inferred using other investigated performance measures. Furthermore, the evidence obtained from the results suggests that the change in number of base learners or type of meta‐learner does not exhibit significant change in the performance of corresponding heterogenous ensemble classifiers.
computer science, software engineering
What problem does this paper attempt to address?