Abstract:Classifier ensemble generally should combine diverse component classifiers. However, it is difficult to give a definitive connection between diversity measure and ensemble accuracy. Given a list of available component classifiers, how to adaptively and diversely ensemble classifiers becomes a big challenge in the literature. In this paper, we argue that diversity, not direct diversity on samples but adaptive diversity with data, is highly correlated to ensemble accuracy, and we propose a novel technology for classifier ensemble, learning to diversify, which learns to adaptively combine classifiers by considering both accuracy and diversity. Specifically, our approach, Learning TO Diversify via Weighted Kernels (L2DWK), performs classifier combination by optimizing a direct but simple criterion: maximizing ensemble accuracy and adaptive diversity simultaneously by minimizing a convex loss function. Given a measure formulation, the diversity is calculated with weighted kernels (i.e., the diversity is measured on the component classifiers' outputs which are kernelled and weighted), and the kernel weights are automatically learned. We minimize this loss function by estimating the kernel weights in conjunction with the classifier weights, and propose a self-training algorithm for conducting this convex optimization procedure iteratively. Extensive experiments on a variety of 32 UCI classification benchmark datasets show that the proposed approach consistently outperforms state-of-the-art ensembles such as Bagging, AdaBoost, Random Forests, Gasen, Regularized Selective Ensemble, and Ensemble Pruning via Semi-Definite Programming.

A Novel Method for Constructing Ensemble Classifiers

A Variant of Rotation Forest for Constructing Ensemble Classifiers

A New Rotation Forest Ensemble Algorithm

Classy Ensemble: A Novel Ensemble Algorithm for Classification

Dynamically building diversified classifier pruning ensembles via canonical correlation analysis

Benchmarking Ensemble Classifiers with Novel Co-Trained Kernal Ridge Regression and Random Vector Functional Link Ensembles [Research Frontier].

A Novel Genetic Algorithm for Subspace Based Subclasssifier Selection

Building Locally Discriminative Classifier Ensemble Through Classifier Fusion among Nearest Neighbors

An Ensemble Classifier Based on Attribute Selection and Diversity Measure

A New Diversity Measure for Classifier Fusion

Novel Ensemble Modeling Method for Enhancing Subset Diversity Using Clustering Indicator Vector Based on Stacked Autoencoder

Integration of Classifier Diversity Measures for Feature Selection-Based Classifier Ensemble Reduction.

BoostTree and BoostForest for Ensemble Learning

Dbboost: Enhancing Imbalanced Classification by A Novel Ensemble Based Technique

A Novel Feature Grouping Method for Ensemble Neural Network Using Localized Generalization Error Model

Ensemble Classification Based On Random Linear Base Classifiers

A Novel Ensemble Method for Classifying Imbalanced Data

Subspace Ensembles for Classification

A Hybrid and Ensemble Intelligent Pattern Classification Algorithm

Learning to Diversify via Weighted Kernels for Classifier Ensemble

Credit Scoring Using Ensemble Classification Based on Variable Weighting Clustering