Abstract:In practice, some bugs have more impact than others and thus deserve more immediate attention. Due to tight schedule and limited human resources, developers may not have enough time to inspect all bugs. Thus, they often concentrate on bugs that are highly impactful. In the literature, high-impact bugs are used to refer to the bugs which appear at unexpected time or locations and bring more unexpected effects (i.e., surprise bugs), or break pre-existing functionalities and destroy the user experience (i.e., breakage bugs). Unfortunately, identifying high-impact bugs from thousands of bug reports in a bug tracking system is not an easy feat. Thus, an automated technique that can identify high-impact bug reports can help developers to be aware of them early, rectify them quickly, and minimize the damages they cause. Considering that only a small proportion of bugs are high-impact bugs, the identification of high-impact bug reports is a difficult task. In this paper, we propose an approach to identify high-impact bug reports by leveraging imbalanced learning strategies. We investigate the effectiveness of various variants, each of which combines one particular imbalanced learning strategy and one particular classification algorithm. In particular, we choose four widely used strategies for dealing with imbalanced data and four state-of-the-art text classification algorithms to conduct experiments on four datasets from four different open source projects. We mainly perform an analytical study on two types of high-impact bugs, i.e., surprise bugs and breakage bugs. The results show that different variants have different performances, and the best performing variants SMOTE (synthetic minority over-sampling technique) + KNN (K-nearest neighbours) for surprise bug identification and RUS (random under-sampling) + NB (naive Bayes) for breakage bug identification outperform the F1-scores of the two state-of-the-art approaches by Thung et al. and Garcia and Shihab.

Manifesting Bugs in Machine Learning Code: An Explorative Study with Mutation Testing

Learning Likely Invariants to Explain Why a Program Fails

What Are We Really Testing in Mutation Testing for Machine Learning? A Critical Reflection

Heuristics for Strongly Killing Mutants in Software Mutation Testing

Mutation Testing in Evolving Systems: Studying the relevance of mutants to code evolution

An Exploratory Study on Using Large Language Models for Mutation Testing

An Empirical Study of Bugs in Machine Learning Systems

Towards Effective Metamorphic Testing by Algorithm Stability for Linear Classification Programs

Testing and Validating Machine Learning Classifiers by Metamorphic Testing.

High-Impact Bug Report Identification with Imbalanced Learning Strategies

Learning from Mutants: Using Code Mutation to Learn and Monitor Invariants of a Cyber-Physical System

Automated Identification of High Impact Bug Reports Leveraging Imbalanced Learning Strategies

MQP: Mutants Quality Prediction for Cost-Effective Mutation Testing

An Empirical Evaluation of Manually Created Equivalent Mutants

DevMuT: Testing Deep Learning Framework Via Developer Expertise-Based Mutation

Test Data Generation for Mutation Testing Based on Markov Chain Usage Model and Estimation of Distribution Algorithm

A Comparative Study of Supervised Learning Algorithms for Re-opened Bug Prediction.

Does mutation testing improve testing practices?

DeepMutation: Mutation Testing of Deep Learning Systems

A new perspective on the competent programmer hypothesis through the reproduction of bugs with repeated mutations

Mutation Testing of Unsupervised Learning Systems