Abstract:AdaBoost has been theoretically and empirically proved to be a very successful ensemble learning algorithm, which iteratively generates a set of diverse weak learners and combines their outputs using the weighted majority voting rule as the final decision. However, in some cases, AdaBoost leads to overfitting especially for mislabeled noisy training examples, resulting in both its degraded generalization performance and non-robustness. Recently, a representative approach named noise-detection based AdaBoost (ND_AdaBoost) has been proposed to improve the robustness of AdaBoost in the two-class classification scenario, however, in the multi-class scenario, this approach can hardly achieve satisfactory performance due to the following three reasons. (1) If we decompose a multi-class classification problem using such strategies as one-versus-all or one-versus-one, the obtained two-class problems usually have imbalanced training sets, which negatively influences the performance of ND_AdaBoost. (2) If we directly apply ND_AdaBoost to the multi-class classification scenario, its two-class loss function is no longer applicable and its accuracy requirement for the (weak) base classifiers, i.e., greater than 0.5, is too strong to be almost satisfied. (3) ND_AdaBoost still has the tendency of overfitting as it increases the weights of correctly classified noisy examples, which could make it focus on learning these noisy examples in the subsequent iterations. To solve the dilemma, in this paper, we propose a robust multi-class AdaBoost algorithm (Rob_MulAda) whose key ingredients consist in a noise-detection based multi-class loss function and a new weight updating scheme. Experimental study indicates that our newly-proposed weight updating scheme is indeed more robust to mislabeled noises than that of ND_AdaBoost in both two-class and multi-class scenarios. In addition, through the comparison experiments, we also verify the effectiveness of Rob_MulAda and provide a suggestion in choosing the most appropriate noise-alleviating approach according to the concrete noise level in practical applications.

Robust Adaptive Label Propagation By Double Matrix Decomposition

Robust Adaptive Embedded Label Propagation with Weight Learning for Inductive Classification

Robust Triple-Matrix-Recovery-Based Auto-Weighted Label Propagation for Classification

A Label Noise Robust Stacked Auto-Encoder Algorithm for Inaccurate Supervised Classification Problems

Adaptive Neighborhood Propagation by Joint L2,1-Norm Regularized Sparse Coding for Representation and Classification

Robust Learning with Adversarial Perturbations and Label Noise: A Two-Pronged Defense Approach.

Dynamic Loss For Robust Learning

Projective Label Propagation by Label Embedding: A Deep Label Prediction Framework for Representation and Classification

Improving deep label noise learning with dual active label correction

Multi-annotator Deep Learning: A Probabilistic Framework for Classification

Bilinear Embedding Label Propagation: Towards Scalable Prediction of Image Labels.

Meta Transition Adaptation for Robust Deep Learning with Noisy Labels

Robust Long-Tailed Learning under Label Noise

Multi-level Label Propagation Algorithm Based on Data Reconstruction

A Robust Multi-Class AdaBoost Algorithm for Mislabeled Noisy Data

Robust Semi-supervised Multi-label Learning by Triple Low-Rank Regularization.

Confidence Regularized Label Propagation Based Domain Adaptation

Hybrid Noise-Oriented Multilabel Learning.

A Novel Probabilistic Label Enhancement Algorithm for Multi-Label Distribution Learning

Semi-supervised imbalanced multi-label classification with label propagation