Abstract:Context: Software defect prediction (SDP) is an important challenge in the field of software engineering, hence much research work has been conducted, most notably through the use of machine learning algorithms. However, class-imbalance typified by few defective components and many non-defective ones is a common occurrence causing difficulties for these methods. Imbalanced learning aims to deal with this problem and has recently been deployed by some researchers, unfortunately with inconsistent results. Objective: We conduct a comprehensive experiment to explore (a) the basic characteristics of this problem; (b) the effect of imbalanced learning and its interactions with (i) data imbalance, (ii) type of classifier, (iii) input metrics and (iv) imbalanced learning method. Method: We systematically evaluate 27 data sets, 7 classifiers, 7 types of input metrics and 17 imbalanced learning methods (including doing nothing) using an experimental design that enables exploration of interactions between these factors and individual imbalanced learning algorithms. This yields 27 x7; 7 x7; 7 x7; 17 = 22491 results. The Matthews correlation coefficient (MCC) is used as an unbiased performance measure (unlike the more widely used F1 and AUC measures). Results: (a) we found a large majority (87 percent) of 106 public domain data sets exhibit moderate or low level of imbalance (imbalance ratio 003C;10; median = 3.94); (b) anything other than low levels of imbalance clearly harm the performance of traditional learning for SDP; (c) imbalanced learning is more effective on the data sets with moderate or higher imbalance, however negative results are always possible; (d) type of classifier has most impact on the improvement in classification performance followed by the imbalanced learning method itself. Type of input metrics is not influential. (e) only ${\sim} 52\%$similar to 52% of the combinations of Imbalanced Learner and Classifier have a significant positive effect. Conclusion: This paper offers two practical guidelines. First, imbalanced learning should only be considered for moderate or highly imbalanced SDP data sets. Second, the appropriate combination of imbalanced method and classifier needs to be carefully chosen to ameliorate the imbalanced learning problem for SDP. In contrast, the indiscriminate application of imbalanced learning can be harmful.

An Improved SDA Based Defect Prediction Framework for Both Within-Project and Cross-Project Class-Imbalance Problems

A Novel Class-Imbalance Learning Approach for Both Within-Project and Cross-Project Defect Prediction.

Cross-Project and Within-Project Semi-Supervised Software Defect Prediction Problems Study Using a Unified Solution

An Improved Transfer Adaptive Boosting Approach for Mixed‐project Defect Prediction

A Comprehensive Investigation of the Role of Imbalanced Learning for Software Defect Prediction

A Hybrid Sampling and Multi-Objective Optimization Approach for Enhanced Software Defect Prediction

Towards Cross-Project Defect Prediction with Imbalanced Feature Sets

DSSDPP: Data Selection and Sampling Based Domain Programming Predictor for Cross-Project Defect Prediction

SDP-MTF: A Composite Transfer Learning and Feature Fusion for Cross-Project Software Defect Prediction

Combined Classifier for Cross-Project Defect Prediction: an Extended Empirical Study.

A novel instance‐based method for cross‐project just‐in‐time defect prediction

An Improved Semi-Supervised Learning Method for Software Defect Prediction.

Tackling Class Imbalance Problem In Software Defect Prediction Through Cluster-Based Over-Sampling With Filtering

Comparative Study of Ensemble Learning Methods in Just-in-time Software Defect Prediction

Class Imbalance Reduction and Centroid based Relevant Project Selection for Cross Project Defect Prediction

Empirical Evaluation of the Impact of Class Overlap on Software Defect Prediction

Software defect prediction based on nested-stacking and heterogeneous feature selection

A Survey of Different Approaches for the Class Imbalance Problem in Software Defect Prediction

Cross-Project Defect Prediction Considering Multiple Data Distribution Simultaneously

A Cross-Project Defect Prediction Model Based on Deep Learning With Self-Attention

UDA-DP: Unsupervised Domain Adaptation for Software Defect Prediction