Abstract:In this paper, sixty-eight research articles published between 2000 and 2017 as well as textbooks which employed four classification algorithms: K-Nearest-Neighbor (KNN), Support Vector Machines (SVM), Random Forest (RF) and Neural Network (NN) as the main statistical tools were reviewed. The aim was to examine and compare these nonparametric classification methods on the following attributes: robustness to training data, sensitivity to changes, data fitting, stability, ability to handle large data sizes, sensitivity to noise, time invested in parameter tuning, and accuracy. The performances, strengths and shortcomings of each of the algorithms were examined, and finally, a conclusion was arrived at on which one has higher performance. It was evident from the literature reviewed that RF is too sensitive to small changes in the training dataset and is occasionally unstable and tends to overfit in the model. KNN is easy to implement and understand but has a major drawback of becoming significantly slow as the size of the data in use grows, while the ideal value of K for the KNN classifier is difficult to set. SVM and RF are insensitive to noise or overtraining, which shows their ability in dealing with unbalanced data. Larger input datasets will lengthen classification times for NN and KNN more than for SVM and RF. Among these nonparametric classification methods, NN has the potential to become a more widely used classification algorithm, but because of their time-consuming parameter tuning procedure, high level of complexity in computational processing, the numerous types of NN architectures to choose from and the high number of algorithms used for training, most researchers recommend SVM and RF as easier and wieldy used methods which repeatedly achieve results with high accuracies and are often faster to implement.

Classification Performance Analysis of Decision Tree-Based Algorithms with Noisy Class Variable

FROD: an Efficient Framework for Optimizing Decision Trees in Packet Classification

A Noise-Resistant Clustering Algorithm for Switching Regression Models

Classification Based on Decision Tree Algorithm for Machine Learning

Performance of Classifiers on Noisy-Labeled Training Data: An Empirical Study on Handwritten Digit Classification Task

A study of the effect of different types of noise on the precision of supervised learning techniques

Statistical analysis of various splitting criteria for decision trees

The Classification of Noise-Afflicted Remotely Sensed Data Using Three Machine-Learning Techniques: Effect of Different Levels and Types of Noise on Accuracy

Robust Loss Functions for Training Decision Trees with Noisy Labels

Taiga: Performance Optimization of the C4.5 Decision Tree Construction Algorithm

Basic Tenets of Classification Algorithms K-Nearest-Neighbor, Support Vector Machine, Random Forest and Neural Network: A Review

A New Method for Learning Decision Tree Classifier

Probabilistic Random Forest: A machine learning algorithm for noisy datasets

Building a Decision Tree Model for Academic Advising Affairs Based on the Algorithm C 4-5

Decoding class dynamics in learning with noisy labels

Improved differentiation classification of variable precision artificial intelligence higher education management

An Improved Network Traffic Classification Algorithm Based on Hadoop Decision Tree

Prediction of Student Performance Using Random Forest Combined With Naïve Bayes

When Fair Classification Meets Noisy Protected Attributes

Exploring Loss Design Techniques For Decision Tree Robustness To Label Noise