Abstract:Traditional ensemble learning approaches explore the feature space and the sample space, respectively, which will prevent them to construct more powerful learning models for noisy real-world dataset classification. The random subspace method only search for the selection of features. Meanwhile, the bagging approach only search for the selection of samples. To overcome these limitations, we propose the hybrid incremental ensemble learning (HIEL) approach which takes into consideration the feature space and the sample space simultaneously to handle noisy dataset. Specifically, HIEL first adopts the bagging technique and linear discriminant analysis to remove noisy attributes, and generates a set of bootstraps and the corresponding ensemble members in the subspaces. Then, the classifiers are selected incrementally based on a classifier-specific criterion function and an ensemble criterion function. The corresponding weights for the classifiers are assigned during the same process. Finally, the final label is summarized by a weighted voting scheme, which serves as the final result of the classification. We also explore various classifier-specific criterion functions based on different newly proposed similarity measures, which will alleviate the effect of noisy samples on the distance functions. In addition, the computational cost of HIEL is analyzed theoretically. A set of nonparametric tests are adopted to compare HIEL and other algorithms over several datasets. The experiment results show that HIEL performs well on the noisy datasets. HIEL outperforms most of the compared classifier ensemble methods on 14 out of 24 noisy real-world UCI and KEEL datasets.

Dynamic Weighting Ensembles for Incremental Learning

An ensemble learning method based on Bagging-SVM for intelligent sensor

Weights Updated Voting for Ensemble of Neural Networks Based Incremental Learning

Dynamically Weighted Ensemble Neural Networks for Regression Problems

An Ensemble of Neural Networks Based Incremental Learning Algorithm with Weights-updated Voting

Incremental Classification Method Based on Ensemble

Incremental Weighted Ensemble for Data Streams With Concept Drift.

Incremental Learning Imbalanced Data Streams with Concept Drift: the Dynamic Updated Ensemble Algorithm

Selective Bagging Based Incremental Learning

Dynamic Ensemble of Ensembles in Nonstationary Environments

Study of Integration Method for Multiple Classifiers on Ensemble Learning

Incremental learning based on ensemble pruning.

Rethinking Class-Incremental Learning from a Dynamic Imbalanced Learning Perspective

Hybrid Incremental Ensemble Learning for Noisy Real-World Data Classification.

Incremental Learning of Support Vector Machines by Classifier Combining

Microcluster-Based Incremental Ensemble Learning for Noisy, Nonstationary Data Streams

A distance-based weighting framework for boosting the performance of dynamic ensemble selection

An Incremental Network with Local Experts Ensemble.

Reinforcement Learning based dynamic weighing of Ensemble Models for Time Series Forecasting

Deep Incremental Boosting

A Class Incremental Learning Method with Weight Re-Initialization and Multiple Distillation for Image Classification