Abstract:<p>The dynamic ensemble selection of classifiers is an effective approach for processing label-imbalanced data classifications. However, such a technique is prone to overfitting, owing to the lack of regularization methods and the dependence of the aforementioned technique on local geometry. In this study, focusing on binary imbalanced data classification, a novel dynamic ensemble method, namely adaptive ensemble of classifiers with regularization (AER), is proposed, to overcome the stated limitations. The method solves the overfitting problem through implicit regularization. Specifically, it leverages the properties of stochastic gradient descent to obtain the solution with the minimum norm, thereby achieving regularization; furthermore, it interpolates the ensemble weights by exploiting the global geometry of data to further prevent overfitting. According to our theoretical proofs, the seemingly complicated AER paradigm, in addition to its regularization capabilities, can actually reduce the asymptotic time and memory complexities of several other algorithms. We evaluate the proposed AER method on seven benchmark imbalanced datasets from the UCI machine learning repository and one artificially generated GMM-based dataset with five variations. The results show that the proposed algorithm outperforms the major existing algorithms based on multiple metrics in most cases, and two hypothesis tests (McNemar's and Wilcoxon tests) verify the statistical significance further. In addition, the proposed method has other preferred properties such as special advantages in dealing with highly imbalanced data, and it pioneers the research on the regularization for dynamic ensemble methods.</p>

A Parallelizable Bayesian Ensemble Online Learning Algorithm

BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning

A Bayesian Approach to (Online) Transfer Learning: Theory and Algorithms

Development and validation of a model that estimates body fat percentage based on simple anthropometric measurements.

Leveraging Linear Independence of Component Classifiers: Optimizing Size and Prediction Accuracy for Online Ensembles

Classification of High-Dimensional Evolving Data Streams Via a Resource-Efficient Online Ensemble

Probabilistic Ensemble of Collaborative Filters

A Novel Surrogate-assisted Evolutionary Algorithm Applied to Partition-based Ensemble Learning

BoostTree and BoostForest for Ensemble Learning

A practical tutorial on bagging and boosting based ensembles for machine learning: Algorithms, software tools, performance study, practical perspectives and opportunities

PBIL ensemble: Many better than one

Evolutionary bagging for ensemble learning

Bayesian Online Learning for Consensus Prediction

Dynamic Online Ensembles of Basis Expansions

A Probabilistic Ensemble Pruning Algorithm

On Optimizing Ensemble Models using Column Generation

Adaptive ensemble of classifiers with regularization for imbalanced data classification

Online Ensemble Approach for Probabilistic Wind Power Forecasting

Online ensemble learning algorithm for imbalanced data stream

A hybrid ensemble and evolutionary algorithm for imbalanced classification and its application on bioinformatics

Online Ensemble Learning for Load Forecasting