Abstract:AdaBoost has been theoretically and empirically proved to be a very successful ensemble learning algorithm, which iteratively generates a set of diverse weak learners and combines their outputs using the weighted majority voting rule as the final decision. However, in some cases, AdaBoost leads to overfitting especially for mislabeled noisy training examples, resulting in both its degraded generalization performance and non-robustness. Recently, a representative approach named noise-detection based AdaBoost (ND_AdaBoost) has been proposed to improve the robustness of AdaBoost in the two-class classification scenario, however, in the multi-class scenario, this approach can hardly achieve satisfactory performance due to the following three reasons. (1) If we decompose a multi-class classification problem using such strategies as one-versus-all or one-versus-one, the obtained two-class problems usually have imbalanced training sets, which negatively influences the performance of ND_AdaBoost. (2) If we directly apply ND_AdaBoost to the multi-class classification scenario, its two-class loss function is no longer applicable and its accuracy requirement for the (weak) base classifiers, i.e., greater than 0.5, is too strong to be almost satisfied. (3) ND_AdaBoost still has the tendency of overfitting as it increases the weights of correctly classified noisy examples, which could make it focus on learning these noisy examples in the subsequent iterations. To solve the dilemma, in this paper, we propose a robust multi-class AdaBoost algorithm (Rob_MulAda) whose key ingredients consist in a noise-detection based multi-class loss function and a new weight updating scheme. Experimental study indicates that our newly-proposed weight updating scheme is indeed more robust to mislabeled noises than that of ND_AdaBoost in both two-class and multi-class scenarios. In addition, through the comparison experiments, we also verify the effectiveness of Rob_MulAda and provide a suggestion in choosing the most appropriate noise-alleviating approach according to the concrete noise level in practical applications.

Semi-supervised multi-class Adaboost by exploiting unlabeled data

Improved multi-class semi-supervised learning with boosting method

Dual-Classifier Collaborative Method Based on Semi-Supervised Active Learning

Multi-class AdaBoost ELM

Imbalanced Data Sets Classification Method Based on Over-Sampling Technique

Two-Stage Multi-Class Adaboost for Facial Expression Recognition

Face Recognition Based on Multi-Adaboost

Boosting Separability in Semisupervised Learning for Object Classification

sEMG multi-class pattern recognition based on semi-supervised boosting algorithm

A Novel Semi-Supervised Adaboost Technique Based On Improved Tri-Training

Semi-Supervised Dual Relation Learning for Multi-Label Classification

Optimally Combining Classifiers for Semi-Supervised Learning

Semi-Supervised Ensemble Classification Method Based On Near Neighbor And Its Application

Transfer Learning via Semi-supervised Boosting Method

AN ADAPTIVE BOOSTING ALGORITHM WITH SEMI-SUPERVISED LEARNING FOR FACIAL EXPRESSION RECOGNITION

Self Adaptive Threshold Pseudo-labeling and Unreliable Sample Contrastive Loss for Semi-supervised Image Classification

Semi-supervised tri-Adaboost algorithm for network intrusion detection

Adaptive Semi-Supervised Learning with Discriminative Least Squares Regression

Boost Picking: A Universal Method on Converting Supervised Classification to Semi-supervised Classification.

Training Boosting-Like Algorithms with Semi-Supervised Subspace Learning

A Robust Multi-Class AdaBoost Algorithm for Mislabeled Noisy Data