Abstract:Background: Unsupervised machine learners have been increasingly applied to software defect prediction. It is an approach that may be valuable for software practitioners because it reduces the need for labeled training data. Objective: Investigate the use and performance of unsupervised learning techniques in software defect prediction. Method: We conducted a systematic literature review that identified 49 studies containing 2456 individual experimental results, which satisfied our inclusion criteria published between January 2000 and March 2018. In order to compare prediction performance across these studies in a consistent way, we (re-)computed the confusion matrices and employed the Matthews Correlation Coefficient (MCC) as our main performance measure. Results: Our meta-analysis shows that unsupervised models are comparable with supervised models for both within-project and cross-project prediction. Among the 14 families of unsupervised model, Fuzzy CMeans (FCM) and Fuzzy SOMs (FSOMs) perform best. In addition, where we were able to check, we found that almost 11% (262/2456) of published results (contained in 16 papers) were internally inconsistent and a further 33% (823/2456) provided insufficient details for us to check. Conclusion: Although many factors impact the performance of a classifier, e.g., dataset characteristics, broadly speaking, unsupervised classifiers do not seem to perform worse than the supervised classifiers in our review. However, we note a worrying prevalence of (i) demonstrably erroneous experimental results, (ii) undemanding benchmarks and (iii) incomplete reporting. We therefore encourage researchers to be comprehensive in their reporting.

Nonparametric Unsupervised Classification

An Unsupervised Learning Classifier with Competitive Error Performance

A novel semi-supervised hyperspectral image classification approach based on spatial neighborhood information and classifier combination

Unsupervised Neighborhood Component Analysis for Clustering.

An ensemble-driven k-NN approach to ill-posed classification problems

Under-bagging Nearest Neighbors for Imbalanced Classification

Information Entropy Based Clustering Method for Unsupervised Internet Traffic Classification

Optimal Scoring for Unsupervised Learning.

Unsupervised K-Nearest Neighbor Regression

Deep Learning with Nonparametric Clustering

Nonparametric semi-supervised learning of class proportions

Unsupervised Deep Discriminant Analysis Based Clustering

A Semi-Supervised Self-Training Ensemble Method Based on Clustering

Data Imputation by Pursuing Better Classification: A Supervised Kernel-Based Method

A Cluster-Based Semisupervised Ensemble for Multiclass Classification.

Can an unsupervised clustering algorithm reproduce a categorization system?

A systematic review of unsupervised learning techniques for software defect prediction

A Novel Classification Algorithm Based on the Synergy Between Dynamic Clustering with Adaptive Distances and K-Nearest Neighbors

Two Novel Kernel-Based Semi-Supervised Clustering Methods By Seeding

A Novel Classification Method: Neighborhood-Based Positive Unlabeled Learning Using Decision Tree (NPULUD)

Fractionally-Supervised Classification