ELBlocker: Predicting blocking bugs with ensemble imbalance learning

Xin Xia,David Lo,Emad Shihab,Xinyu Wang,Xiaohu Yang

DOI: https://doi.org/10.1016/j.infsof.2014.12.006

IF: 3.9

2015-01-01

Information and Software Technology

Abstract:•We propose a ELBlocker to predict the blocking bugs.•We evaluate ELBlocker on 6 large software projects.•Experiment results show our approach achieves substantial improvements over these methods.

What problem does this paper attempt to address?

Predicting the Breakability of Blocking Bug Pairs

Hui Ding,Wanwangying Ma,Lin Chen,Yuming Zhou,Baowen Xu

DOI: https://doi.org/10.1109/COMPSAC.2018.00035

2018-01-01

Abstract:Software systems becomes increasingly complex for the wide use of social-software-development platforms, such as GitHub. Complicated inter-dependencies within ecosystems impose new challenges in resolving the blocking bug pairs, in which the upstream bugs prevent the downstream bugs to be fixed. Generally, the downstream bugs cannot be fixed until the upstream bugs are fixed in blocking bug pairs, which keeps the downstream developers waiting for a long time. However, previous research found that some blocking pairs could be "broken" through a workaround, i.e., a temporary solution proposed by the downstream developers before the upstream bugs get fixed. In this paper, we propose an approach to describe and predict the breakability of the blocking bug pairs. Our goal is to help the ecosystem developers to predict whether the given blocking bug pair can be broken. We evaluate our approach on two real world ecosystems: Mozilla Firefox and Netbeans, and have the following two main findings. First, the participants within the blocking bug pair and the vitality of the downstream bug have much impact on the breakability. Second, we can build breakability predication models with the accuracy of approximate 80%. Moreover, for predicting unbreakable pairs which affect the downstream projects seriously, our approach achieves a precision over 92%. It can be used to effectively remind the upstream developers to fix these bugs as quickly as possible.
Predicting concurrency bugs: how many, what kind and where are they?

Bo Zhou,Iulian Neamtiu,Rajiv Gupta

DOI: https://doi.org/10.1145/2745802.2745807

2015-01-01

Abstract:Concurrency bugs are difficult to find and fix. To help with finding and fixing concurrency bugs, prior research has mostly focused on static or dynamic analyses for finding specific classes of bugs. We present an approach whose focus is understanding the differences between concurrency and non-concurrency bugs, the differences among various concurrency bug classes, and predicting bug quantity, type, and location, from patches, bug reports and bug-fix metrics. First, we show that bug characteristics and bug-fixing processes vary significantly among different kinds of concurrency bugs and compared to non-concurrency bugs. Next, we build a quantitative predictor model to estimate concurrency bugs appearance in future releases. Then, we build a qualitative predictor that can predict the type of concurrency bug for a newly-filed bug report. Finally, we build a bug location predictor to indicate the likely source code location for newly-reported bugs. We validate the effectiveness of our approach on three popular projects, Mozilla, KDE, and Apache.
Automatic, High Accuracy Prediction of Reopened Bugs

Xin Xia,David Lo,Emad Shihab,Xinyu Wang,Bo Zhou

DOI: https://doi.org/10.1007/s10515-014-0162-2

IF: 1.677

2014-01-01

Automated Software Engineering

Abstract:Bug fixing is one of the most time-consuming and costly activities of the software development life cycle. In general, bugs are reported in a bug tracking system, validated by a triage team, assigned for someone to fix, and finally verified and closed. However, in some cases bugs have to be reopened. Reopened bugs increase software maintenance cost, cause rework for already busy developers and in some cases even delay the future delivery of a software release. Therefore, a few recent studies focused on studying reopened bugs. However, these prior studies did not achieve high performance (in terms of precision and recall), required manual intervention, and used very simplistic techniques when dealing with this textual data, which leads us to believe that further improvements are possible. In this paper, we propose ReopenPredictor, which is an automatic, high accuracy predictor of reopened bugs. ReopenPredictor uses a number of features, including textual features, to achieve high accuracy prediction of reopened bugs. As part of ReopenPredictor, we propose two algorithms that are used to automatically estimate various thresholds to maximize the prediction performance. To examine the benefits of ReopenPredictor, we perform experiments on three large open source projects-namely Eclipse, Apache HTTP and OpenOffice. Our results show that ReopenPredictor outperforms prior work, achieving a reopened F-measure of 0.744, 0.770, and 0.860 for Eclipse, Apache HTTP and OpenOffice, respectively. These results correspond to an improvement in the reopened F-measure of the method proposed in the prior work by Shihab et al. by 33.33, 12.57 and 3.12% for Eclipse, Apache HTTP and OpenOffice, respectively.
Automated Bug Report Field Reassignment and Refinement Prediction

Xin Xia,David Lo,Emad Shihab,Xinyu Wang

DOI: https://doi.org/10.1109/tr.2015.2484074

IF: 5.883

2015-01-01

IEEE Transactions on Reliability

Abstract:Bug fixing is one of the most important activities in software development and maintenance. Bugs are reported, recorded, and managed in bug tracking systems such as Bugzilla. In general, a bug report contains many fields, such as product, component, severity, priority, fixer, operating system (OS), and platform, which provide important information for the bug triaging and fixing process. Our previous study finds that approximately 80% of bug reports have their fields reassigned and refined at least once, and bugs with reassigned and refined fields take more time to fix than bugs with no reassigned and refined fields. Thus, automatically predicting which bug report fields get reassigned and refined could help developers to save bug fixing time. Considering that a bug report could have multiple field re-assignments and refinements (e.g., the product, component, fixer, and other fields of a bug report can get reassigned and refined), in this paper, we propose a multi-label learning algorithm to predict which bug report fields might be reassigned and refined. We note that the number of bug reports with some types of reassignment and refinement (e.g., bugs whose severity fields gets reassigned and refined) is a small proportion of the whole bug report collection, indicating the class imbalance problem. Thus, we propose imbalanced ML.KNN (Im-ML.KNN), which extends ML.KNN, one of the state-of-the-art multi-label learning algorithms, to achieve better performance. Im-ML.KNN is a composite model that combines 3 multi-label classifiers built using different types of features (i.e., meta, textual, and mixed features). We evaluate our solution on 4 large bug report datasets including OpenOffice, Netbeans, Eclipse, and Mozilla containing a total of 190,558 bug reports. We show that Im-ML.KNN can achieve an average F-measure score of 0.56-0.62. We also compare Im-ML.KNN with other state-of-art methods, such as the method proposed by Lamkanfi et al., ML.KNN, and HOMER-NB. The results show that Im-ML.KNN, on average, improves the average F-measure scores of Lamkanfi et al.'s method, ML.KNN, and HOMER-NB by 119.69%, 9.11%, and 161.08%, respectively.
EFSPredictor: Predicting Configuration Bugs with Ensemble Feature Selection.

Bowen Xu,David Lo,Xin Xia,Ashish Sureka,Shanping Li

DOI: https://doi.org/10.1109/apsec.2015.38

2015-01-01

Abstract:The configuration of a system determines the system behavior and wrong configuration settings can adversely impact system's availability, performance, and correctness. We refer to these wrong configuration settings as configuration bugs. The importance of configuration bugs has prompted many researchers to study it, and past studies can be grouped into three categories: detection, localization, and fixing of configuration bugs. In the work, we focus on the detection of configuration bugs, in particular, we follow the line-of-work that tries to predict if a bug report is caused by a wrong configuration setting. Automatically prediction of whether a bug is a configuration bug can help developers reduce debugging effort. We propose a novel approach named EFSPredictor which applies ensemble feature selection on the natural-language description of a bug report. It uses different feature selection approaches (e.g., ChiSquare, GainRatio and Relief) which output different ranked lists of textual features. Next, to obtain a set of representative textual features, EFSPredictor first assigns different scores to the features outputted by these feature selection approaches. Next, for each feature, EFSPredictor sums up the scores outputted by the multiple ranked lists, and outputs the top features (e.g., 25% of the total number of features) as the selected features. Finally, EFSPredictor builds a prediction model based on the selected features. We conduct experiments on 5 bug report datasets (i.e., accumulo, activemq, camel, flume, and wicket) containing a total of 3,203 bugs. The experiment results show that, on average across the 5 projects, EFSPredictor achieves an F1-score to 0.57, which improves the state-of-the-art approach proposed by Xia et al. by 14%.
TLEL: A Two-Layer Ensemble Learning Approach for Just-in-time Defect Prediction

Xinli Yang,David Lo,Xin Xia,Jianling Sun

DOI: https://doi.org/10.1016/j.infsof.2017.03.007

IF: 3.9

2017-01-01

Information and Software Technology

Abstract:Context: Defect prediction is a very meaningful topic, particularly at change-level. Change-level defect prediction, which is also referred as just-in-time defect prediction, could not only ensure software quality in the development process, but also make the developers check and fix the defects in time [1].Objective: Ensemble learning becomes a hot topic in recent years. There have been several studies about applying ensemble learning to defect prediction [2-5]. Traditional ensemble learning approaches only have one layer, i.e., they use ensemble learning once. There are few studies that leverages ensemble learning twice or more. To bridge this research gap, we try to hybridize various ensemble learning methods to see if it will improve the performance of just-in-time defect prediction. In particular, we focus on one way to do this by hybridizing bagging and stacking together and leave other possibly hybridization strategies for future work.Method: In this paper, we propose a two-layer ensemble learning approach TLEL which leverages decision tree and ensemble learning to improve the performance of just-in-time defect prediction. In the inner layer, we combine decision tree and bagging to build a Random Forest model. In the outer layer, we use random under-sampling to train many different Random Forest models and use stacking to ensemble them once more.Results: To evaluate the performance of TLEL, we use two metrics, i.e., cost effectiveness and F1-score. We perform experiments on the datasets from six large open source projects, i.e., Bugzilla, Columba, JDT, Platform, Mozilla, and PostgreSQL, containing a total of 137,417 changes. Also, we compare our approach with three baselines, i.e., Deeper, the approach proposed by us [6], DNC, the approach proposed by Wang et al. [2], and MKEL, the approach proposed by Wang et al. [3]. The experimental results show that on average across the six datasets, TLEL could discover over 70% of the bugs by reviewing only 20% of the lines of code, as compared with about 50% for the baselines. In addition, the F1-scores TLEL can achieve are substantially and statistically significantly higher than those of three baselines across the six datasets.Conclusion: TLEL can achieve a substantial and statistically significant improvement over the state-of-the-art methods, i.e., Deeper, DNC and MKEL. Moreover, TLEL could discover over 70% of the bugs by reviewing only 20% of the lines of code. (C) 2017 Elsevier B.V. All rights reserved.
Automated Identification of High Impact Bug Reports Leveraging Imbalanced Learning Strategies

Xinli Yang,David Lo,Qiao Huang,Xin Xia,Jianling Sun

DOI: https://doi.org/10.1109/compsac.2016.67

2016-01-01

Abstract:In practice, some bugs have more impact than others and thus deserve more immediate attention. Due to tight schedule and limited human resource, developers may not have enough time to inspect all bugs. Thus, they often concentrate on bugs that are highly impactful. In the literature, high impact bugs are used to refer to the bugs which appear in unexpected time or locations and bring more unexpected effects, or break pre-existing functionalities and destroy the user experience. Unfortunately, identifying high impact bugs from the thousands of bug reports in a bug tracking system is not an easy feat. Thus, an automated technique that can identify high-impact bug reports can help developers to be aware of them early, rectify them quickly, and minimize the damages they cause. Considering that only a small proportion of bugs are high impact bugs, the identification of high impact bug reports is a difficult task. In this paper, we propose an approach to identify high impact bug reports by leveraging imbalanced learning strategies. We investigate the effectiveness of various imbalanced learning strategies built upon a number of well-known classification algorithms. In particular, we choose four widely used strategies for dealing with imbalanced data and use naive Bayes multinominal as the classification algorithm to conduct experiments on four datasets from four different open source projects. We perform an empirical study on a specific type of high impact bugs, i.e., surprise bugs, which were first studied by Shihab et al. The results show that under-sampling is the best imbalanced learning strategy with naive Bayes multinominal for high impact bug identification.
Combined Classifier for Cross-Project Defect Prediction: an Extended Empirical Study.

Yun Zhang,David Lo,Xin Xia,Jianling Sun

DOI: https://doi.org/10.1007/s11704-017-6015-y

IF: 2.6688

2018-01-01

Frontiers of Computer Science

Abstract:To facilitate developers in effective allocation of their testing and debugging efforts, many software defect prediction techniques have been proposed in the literature. These techniques can be used to predict classes that are more likely to be buggy based on the past history of classes, methods, or certain other code elements. These techniques are effective provided that a sufficient amount of data is available to train a prediction model. However, sufficient training data are rarely available for new software projects. To resolve this problem, cross-project defect prediction, which transfers a prediction model trained using data from one project to another, was proposed and is regarded as a new challenge in the area of defect prediction. Thus far, only a few cross-project defect prediction techniques have been proposed. To advance the state of the art, in this study, we investigated seven composite algorithms that integrate multiple machine learning classifiers to improve cross-project defect prediction. To evaluate the performance of the composite algorithms, we performed experiments on 10 open-source software systems from the PROMISE repository, which contain a total of 5,305 instances labeled as defective or clean. We compared the composite algorithms with the combined defect predictor where logistic regression is used as the meta classification algorithm (CODEP (Logistic) ), which is the most recent cross-project defect prediction algorithm in terms of two standard evaluation metrics: cost effectiveness and F-measure. Our experimental results show that several algorithms outperform CODEP (Logistic) : Maximum voting shows the best performance in terms of F-measure and its average F-measure is superior to that of CODEP (Logistic) by 36.88%. Bootstrap aggregation (Bagging (J48)) shows the best performance in terms of cost effectiveness and its average cost effectiveness is superior to that of CODEP (Logistic) by 15.34%.
An Empirical Study on Critical Blocking Bugs.

Hao Ren,Yanhui Li,Lin Chen

DOI: https://doi.org/10.1145/3387904.3389267

2020-01-01

Abstract:Blocking bugs are a severe type of bugs that prevent other bugs from being fixed. As software becomes increasingly complex and large, blocking bugs occur in many large-scale software, especially in software ecosystems. Blocking bugs may have a high negative impact on software development and maintenance. Usually, blocking bugs preventing more bugs should be more concerned. In this paper, we focus on a special type of blocking bugs that block at least two bugs, which we call Critical Blocking Bugs (CBBs). We study CBBs from the following five aspects: the importance, the repair time, the scale of repair, the experience of developers who repair CBBs, and the circumstance why CBBs block multiple bugs. We build a dataset containing five open source projects and classify bugs into three types (i.e., critical blocking bugs, normal blocking bugs and other bugs, which block at least two, just one and zero bugs, respectively) to compare the differences between CBBs and other types of bugs. The experimental results show that CBBs are more important with longer repair time and larger repair scale, and CBBs are concentrated on parts of components of the project. These results highlight that CBBs are different from other types of bugs in many aspects, and we should pay more attention to such bugs in the future software maintenance process.
High-Impact Bug Report Identification with Imbalanced Learning Strategies

Xin-Li Yang,David Lo,Xin Xia,Qiao Huang,Jian-Ling Sun

DOI: https://doi.org/10.1007/s11390-017-1713-3

IF: 1.871

2017-01-01

Journal of Computer Science and Technology

Abstract:In practice, some bugs have more impact than others and thus deserve more immediate attention. Due to tight schedule and limited human resources, developers may not have enough time to inspect all bugs. Thus, they often concentrate on bugs that are highly impactful. In the literature, high-impact bugs are used to refer to the bugs which appear at unexpected time or locations and bring more unexpected effects (i.e., surprise bugs), or break pre-existing functionalities and destroy the user experience (i.e., breakage bugs). Unfortunately, identifying high-impact bugs from thousands of bug reports in a bug tracking system is not an easy feat. Thus, an automated technique that can identify high-impact bug reports can help developers to be aware of them early, rectify them quickly, and minimize the damages they cause. Considering that only a small proportion of bugs are high-impact bugs, the identification of high-impact bug reports is a difficult task. In this paper, we propose an approach to identify high-impact bug reports by leveraging imbalanced learning strategies. We investigate the effectiveness of various variants, each of which combines one particular imbalanced learning strategy and one particular classification algorithm. In particular, we choose four widely used strategies for dealing with imbalanced data and four state-of-the-art text classification algorithms to conduct experiments on four datasets from four different open source projects. We mainly perform an analytical study on two types of high-impact bugs, i.e., surprise bugs and breakage bugs. The results show that different variants have different performances, and the best performing variants SMOTE (synthetic minority over-sampling technique) + KNN (K-nearest neighbours) for surprise bug identification and RUS (random under-sampling) + NB (naive Bayes) for breakage bug identification outperform the F1-scores of the two state-of-the-art approaches by Thung et al. and Garcia and Shihab.
Combining Software Metrics and Text Features for Vulnerable File Prediction

Yun Zhang,David Lo,Xin Xia,Bowen Xu,Jianling Sun,Shanping Li

DOI: https://doi.org/10.1109/ICECCS.2015.15

2015-01-01

Abstract:In recent years, to help developers reduce time and effort required to build highly secure software, a number of prediction models which are built on different kinds of features have been proposed to identify vulnerable source code files. In this paper, we propose a novel approach VULPREDICTOR to predict vulnerable files, it analyzes software metrics and text mining together to build a composite prediction model. VULPREDICTOR first builds 6 underlying classifiers on a training set of vulnerable and non-vulnerable files represented by their software metrics and text features, and then constructs a meta classifier to process the outputs of the 6 underlying classifiers. We evaluate our solution on datasets from three web applications including Drupal, PHPMyAdmin and Moodle which contain a total of 3,466 files and 223 vulnerabilities. The experiment results show that VULPREDICTOR can achieve F1 and EffectivenessRatio@20% scores of up to 0.683 and 75%, respectively. On average across the 3 projects, VULPREDICTOR improves the F1 and EffectivenessRatio@20% scores of the best performing state-of-the-art approaches proposed by Walden et al. by 46.53% and 14.93%, respectively.
A Comparative Study of Supervised Learning Algorithms for Re-opened Bug Prediction.

Xin Xia,David Lo,Xinyu Wang,Xiaohu Yang,Shanping Li,Jianling Sun

DOI: https://doi.org/10.1109/csmr.2013.43

2013-01-01

Abstract:Bug fixing is a time-consuming and costly job which is performed in the whole life cycle of software development and maintenance. For many systems, bugs are managed in bug management systems such as Bugzilla. Generally, the status of a typical bug report in Bugzilla changes from new to assigned, verified and closed. However, some bugs have to be reopened. Reopened bugs increase the software development and maintenance cost, increase the workload of bug fixers, and might even delay the future delivery of a software. Only a few studies investigate the phenomenon of reopened bug reports. In this paper, we evaluate the effectiveness of various supervised learning algorithms to predict if a bug report would be reopened. We choose 7 state-of-the-art classical supervised learning algorithm in machine learning literature, i.e., kNN, SVM, Simple Logistic, Bayesian Network, Decision Table, CART and LWL, and 3 ensemble learning algorithms, i.e., AdaBoost, Bagging and Random Forest, and evaluate their performance in predicting reopened bug reports. The experiment results show that among the 10 algorithms, Bagging and Decision Table (IDTM) achieve the best performance. They achieve accuracy scores of 92.91% and 92.80%, respectively, and reopened bug reports F-Measure scores of 0.735 and 0.732, respectively. These results improve the reopened bug reports F-Measure of the state-of-the-art approaches proposed by Shihab et al. by up to 23.53%.
Why and How Bug Blocking Relations Are Breakable: an Empirical Study on Breakable Blocking Bugs

Hao Ren,Yanhui Li,Lin Chen,Yuming Zhou,Changhai Nie

DOI: https://doi.org/10.1016/j.infsof.2023.107354

IF: 3.9

2024-01-01

Information and Software Technology

Abstract:Blocking bugs prevents other bugs from being fixed, which is difficult to repair and negatively impacts software quality. During software maintenance, developers usually try to break the blocking relationship between blocking and blocked bugs, e.g., propose a temporary fix. However, to our knowledge, no studies have investigated why and how blocking relations between bugs are breakable. In this study, we aim to construct an empirical analysis to explore breakable blocking bugs (BBBs). Specifically, we employ quantitative and qualitative analysis to study these BBBs from two aspects. One is to investigate the characteristics of these bugs, and the other is to explore why and how developers break the blocking relationship between bugs during software maintenance. We build a dataset on five large-scale open-source projects and classify bugs into three types (BBBs, normal blocking bugs, and other bugs) to compare the differences between BBBs and other types of bugs. We observe that BBBs have higher levels of involvement, take longer to fix, and involve more complex source code than other bugs. Moreover, we summarize four reasons blocking relationships between bugs are broken, i.e., partial association (41.87%), serious influence (26.40%), time pressure (19.73%), and flawed blocking (12.21%), and three measures developers adopt to break these blocking relationships, i.e., quick patch for blocking bugs (41.33%), quick patch for blocked bugs (38.67%), and ignore the blocking relation and fix blocked bugs directly (20.00%). Through these analyses, it is meaningful for software maintainers to have a deeper understanding of the characteristics and repair practices of BBBs, which will help solve these BBBs effectively in the future.
Predictive Blacklisting as an Implicit Recommendation System

Fabio Soldo,Anh Le,Athina Markopoulou

DOI: https://doi.org/10.48550/arXiv.0908.2007

2009-08-14

Abstract:A widely used defense practice against malicious traffic on the Internet is through blacklists: lists of prolific attack sources are compiled and shared. The goal of blacklists is to predict and block future attack sources. Existing blacklisting techniques have focused on the most prolific attack sources and, more recently, on collaborative blacklisting. In this paper, we formulate the problem of forecasting attack sources (also referred to as predictive blacklisting) based on shared attack logs as an implicit recommendation system. We compare the performance of existing approaches against the upper bound for prediction, and we demonstrate that there is much room for improvement. Inspired by the recent Netflix competition, we propose a multi-level prediction model that is adjusted and tuned specifically for the attack forecasting problem. Our model captures and combines various factors, namely: attacker-victim history (using time-series) and attackers and/or victims interactions (using neighborhood models). We evaluate our combined method on one month of logs from <a class="link-external link-http" href="http://Dshield.org" rel="external noopener nofollow">this http URL</a> and demonstrate that it improves significantly the state-of-the-art.

Networking and Internet Architecture
Software defect prediction ensemble learning algorithm based on 2-step sparrow optimizing extreme learning machine

Yu Tang,Qi Dai,Mengyuan Yang,Lifang Chen,Ye Du

DOI: https://doi.org/10.1007/s10586-024-04446-y

2024-05-19

Cluster Computing

Abstract:Software defect prediction is a crucial discipline within the software development life cycle. Accurate identification of defective modules in software can result in time and cost savings for developers. The ELM algorithm offers the benefits of rapid training and robust learning capabilities. Numerous researchers in the field of software defect prediction have employed the ELM algorithm. However, the ELM algorithm, a single hidden layer feedforward neural network, faces challenges related to random parameter selection and limited generalization ability. To enhance the predictive performance of the ELM algorithm in software defect prediction. Most researchers utilize swarm intelligence optimization algorithms to optimize extreme learning machines. However, these optimization methods may encounter challenges related to fall into local optimal solution. This paper introduces a new sparrow search algorithm (2SSSA) built upon the original sparrow search algorithm. To enhance the original sparrow algorithm's ability to escape local extrema, the pinhole imaging reverse learning and somersault foraging strategies are employed. The performance of 2SSSA in terms of optimization and convergence speed is assessed using 8 randomly selected benchmark functions and 8 CEC2017 functions. Additionally, ensemble learning is a prominent research focus in the field of software defect prediction. Ensemble learning is known for its ability to significantly enhance prediction performance and model generalization. As a result, the ELM optimized using 2SSSA serves as the foundational predictor in the bagging ensemble learning algorithm. We propose an ensemble algorithm for software defect prediction, denoted as 2SSEBA, which employs a 2-step optimization sparrow algorithm (2SSSA) to optimize extreme learning machines. Based on an evaluation of 25 publicly available software defect prediction datasets using 5 commonly employed metrics. The predictive performance of 2SSEBA significantly outperforms the other five advanced prediction algorithms. Furthermore, this conclusion is supported by both Friedman ranking and Holm's post-hoc test.

computer science, information systems, theory & methods
A comprehensible analysis of the efficacy of Ensemble Models for Bug Prediction

Ingrid Marçal,Rogério Eduardo Garcia

2023-10-19

Abstract:The correctness of software systems is vital for their effective operation. It makes discovering and fixing software bugs an important development task. The increasing use of Artificial Intelligence (AI) techniques in Software Engineering led to the development of a number of techniques that can assist software developers in identifying potential bugs in code. In this paper, we present a comprehensible comparison and analysis of the efficacy of two AI-based approaches, namely single AI models and ensemble AI models, for predicting the probability of a Java class being buggy. We used two open-source Apache Commons Project's Java components for training and evaluating the models. Our experimental findings indicate that the ensemble of AI models can outperform the results of applying individual AI models. We also offer insight into the factors that contribute to the enhanced performance of the ensemble AI model. The presented results demonstrate the potential of using ensemble AI models to enhance bug prediction results, which could ultimately result in more reliable software systems.

Software Engineering,Artificial Intelligence
Robust and accurate performance anomaly detection and prediction for cloud applications: a novel ensemble learning-based framework

Ruyue Xin,Hongyun Liu,Peng Chen,Zhiming Zhao

DOI: https://doi.org/10.1186/s13677-022-00383-6

2023-01-15

Journal of Cloud Computing

Abstract:Effectively detecting run-time performance anomalies is crucial for clouds to identify abnormal performance behavior and forestall future incidents. To be used for real-world applications, an effective anomaly detection framework should meet three main challenging requirements: high accuracy for identifying anomalies, good robustness when application patterns change, and prediction ability for upcoming anomalies. Unfortunately, existing research about performance anomaly detection usually focuses on improving detection accuracy, while little research tackles the three challenges simultaneously. We conduct experiments for existing detection methods on multiple application monitoring data, and results show that existing detection methods usually focus on different features in data, which will lead to their diverse performance on different data patterns. Therefore, existing anomaly detection methods have difficulty improving detection accuracy and robustness and predicting anomalies. To address the three requirements, we propose an Ensemble Learning-Based Detection (ELBD) framework which integrates existing well-selected detection methods. The framework includes three classic linear ensemble methods (maximum, average, and weighted average) and a novel deep ensemble method. Our experiments show that the ELBD framework realizes better detection accuracy and robustness, where the deep ensemble method can achieve the most accurate and robust detection for cloud applications. In addition, it can predict anomalies in the next four minutes with an F1 score higher than 0.8. The paper also proposes a new indicator to measure detection accuracy, robustness, and multi-step prediction ability. The of the deep ensemble method is 5.1821, which is much higher than other detection methods.
Comparative Study of Ensemble Learning Methods in Just-in-time Software Defect Prediction

Ming Wen,Wenjing Zhang,Ruiqi He,Yong Li

DOI: https://doi.org/10.1109/QRS-C60940.2023.00059

2023-10-22

Abstract:Just-in-time software defect prediction (JIT-SDP) is a key way to ensure software safety and quality, and it is receiving increasing attention in the field of software engineering. However, existing JIT-SDP methods have insufficient performance and suffer from imbalanced data classes. To solve these problems, this paper proposes a just-in-time software defect prediction method with class imbalance based on Ensemble learning. Firstly, select four common Oversampling methods (RandomOverSampler, SMOTE, Borderline-SMOTE, and ADASYN) and four classic Ensemble learning methods (Bagging, Random Forest, AdaBoost, and GBDT). In this paper, each Oversampling method is combined with each Ensemble learning method to form 16 different combinations. Secondly, for each combination of raw JIT-SDP data, n imbalanced classes are performed to obtain n-balanced datasets. Then, based on the n balanced data sets, the Ensemble learning algorithm is used to train n independent JIT software defect prediction models. Finally, combine these n models to get an Ensemble learning model to predict new code changes. The experimental results show that the RandomOverSampler has more advantages in dealing with class imbalance. Compared with other combinations, its combination with the Ensemble learning method RF performs best and has better defect prediction performance. It is suitable for JIT-SDP problems in practical applications. When the number of base models is 15, it has the best model performance and computing resources.

Computer Science
Towards Universal Dense Blocking for Entity Resolution

Tianshu Wang,Hongyu Lin,Xianpei Han,Xiaoyang Chen,Boxi Cao,Le Sun

2024-04-25

Abstract:Blocking is a critical step in entity resolution, and the emergence of neural network-based representation models has led to the development of dense blocking as a promising approach for exploring deep semantics in blocking. However, previous advanced self-supervised dense blocking approaches require domain-specific training on the target domain, which limits the benefits and rapid adaptation of these methods. To address this issue, we propose UniBlocker, a dense blocker that is pre-trained on a domain-independent, easily-obtainable tabular corpus using self-supervised contrastive learning. By conducting domain-independent pre-training, UniBlocker can be adapted to various downstream blocking scenarios without requiring domain-specific fine-tuning. To evaluate the universality of our entity blocker, we also construct a new benchmark covering a wide range of blocking tasks from multiple domains and scenarios. Our experiments show that the proposed UniBlocker, without any domain-specific learning, significantly outperforms previous self- and unsupervised dense blocking methods and is comparable and complementary to the state-of-the-art sparse blocking methods.

Databases,Computation and Language,Information Retrieval
Studying the effectiveness of deep active learning in software defect prediction

Farid Feyzi,Arman Daneshdoost

DOI: https://doi.org/10.1080/1206212X.2023.2252117

2023-09-07

International Journal of Computers and Applications

Abstract:Accurate prediction of defective software modules is of great importance for prioritizing quality assurance efforts, reasonably allocating testing resources, reducing costs and improving software quality. Several studies have used machine learning to predict software defects. However, complex structures and imbalanced class distributions in software defect data make learning an effective defect prediction model challenging. In this article, two deep learning-based defect prediction models using static code metrics are proposed. In order to enhance the learning process and improve the performance of the proposed models, pool-based active learning is employed. In this regard, the possibility of using active learning to mitigate the need for a large amount of labeled data in the process of building deep learning models is investigated. To deal with imbalanced distribution of software modules between defective and non-defective classes, Near-Miss under-sampling and KNN, with different number of neighbors, are used. The reason for choosing them is their good performance in binary classification problems. Experiments are performed on two well-known, publicly available datasets, GitHub Bug Dataset and public Unified Bug Dataset for java projects. The evaluation results reveal the effectiveness of our proposed models in comparison to the traditional machine learning algorithms. In the conducted investigations on the Unified Bug Dataset, at the file level, the value of F-measure and AUC criteria have improved by 13 and 11 percent, respectively and at the class level, the values have improved by 14 and 11 percent, respectively.

ELBlocker: Predicting blocking bugs with ensemble imbalance learning

Predicting the Breakability of Blocking Bug Pairs

Predicting concurrency bugs: how many, what kind and where are they?

Automatic, High Accuracy Prediction of Reopened Bugs

Automated Bug Report Field Reassignment and Refinement Prediction

EFSPredictor: Predicting Configuration Bugs with Ensemble Feature Selection.

TLEL: A Two-Layer Ensemble Learning Approach for Just-in-time Defect Prediction

Automated Identification of High Impact Bug Reports Leveraging Imbalanced Learning Strategies

Combined Classifier for Cross-Project Defect Prediction: an Extended Empirical Study.

An Empirical Study on Critical Blocking Bugs.

High-Impact Bug Report Identification with Imbalanced Learning Strategies

Combining Software Metrics and Text Features for Vulnerable File Prediction

A Comparative Study of Supervised Learning Algorithms for Re-opened Bug Prediction.

Why and How Bug Blocking Relations Are Breakable: an Empirical Study on Breakable Blocking Bugs

Predictive Blacklisting as an Implicit Recommendation System

Software defect prediction ensemble learning algorithm based on 2-step sparrow optimizing extreme learning machine

A comprehensible analysis of the efficacy of Ensemble Models for Bug Prediction

Robust and accurate performance anomaly detection and prediction for cloud applications: a novel ensemble learning-based framework

Comparative Study of Ensemble Learning Methods in Just-in-time Software Defect Prediction

Towards Universal Dense Blocking for Entity Resolution

Studying the effectiveness of deep active learning in software defect prediction