Abstract:<p>The dynamic ensemble selection of classifiers is an effective approach for processing label-imbalanced data classifications. However, such a technique is prone to overfitting, owing to the lack of regularization methods and the dependence of the aforementioned technique on local geometry. In this study, focusing on binary imbalanced data classification, a novel dynamic ensemble method, namely adaptive ensemble of classifiers with regularization (AER), is proposed, to overcome the stated limitations. The method solves the overfitting problem through implicit regularization. Specifically, it leverages the properties of stochastic gradient descent to obtain the solution with the minimum norm, thereby achieving regularization; furthermore, it interpolates the ensemble weights by exploiting the global geometry of data to further prevent overfitting. According to our theoretical proofs, the seemingly complicated AER paradigm, in addition to its regularization capabilities, can actually reduce the asymptotic time and memory complexities of several other algorithms. We evaluate the proposed AER method on seven benchmark imbalanced datasets from the UCI machine learning repository and one artificially generated GMM-based dataset with five variations. The results show that the proposed algorithm outperforms the major existing algorithms based on multiple metrics in most cases, and two hypothesis tests (McNemar's and Wilcoxon tests) verify the statistical significance further. In addition, the proposed method has other preferred properties such as special advantages in dealing with highly imbalanced data, and it pioneers the research on the regularization for dynamic ensemble methods.</p>

Optimal Linear Combination Using Decision Reliability Of Individual Classifiers

Optimal Linear Combination of Classifiers

Methods of Linear Combination Based on Different Features

Automatic Coefficient Selection in Weighted Maximum Margin Criterion

A new probability adjustment method for combining conflicting evidences: application in classifier combination

Combination of linear classifiers using score function -- analysis of possible combination strategies

Optimal Classifier Fusion in a Non-Bayesian Probabilistic Framework

Linear Regression Based Combination of Neural Classifiers.

Combination Weighting Approach Based on Deviation Function and Joint-Entropy

Combining classifiers with particle swarms

Dynamic Classifier Combination Method Based on Minimum Cost Criterion

Classifier Combination Based on Active Learning

Decision Combination Based on the Characterisation of Predictive Accuracy.

A distribution-free smoothed combination method to improve discrimination accuracy in multi-category classification

Optimal Portfolio Choice under Decision-Based Model Combinations

Combining information from independent sources through confidence distributions

A Multiple Classifiers Integration Method Based on Adaptive Weigh Adjusting and it's Application on Text Classification

A method of combining multiple probabilistic classifiers through soft competition on different feature sets

Adaptive ensemble of classifiers with regularization for imbalanced data classification

Hybrid SVM algorithm oriented to classifying imbalanced datasets

Estimating individualized optimal combination therapies through outcome weighted deep learning algorithms