Abstract:Classifier ensemble generally should combine diverse component classifiers. However, it is difficult to give a definitive connection between diversity measure and ensemble accuracy. Given a list of available component classifiers, how to adaptively and diversely ensemble classifiers becomes a big challenge in the literature. In this paper, we argue that diversity, not direct diversity on samples but adaptive diversity with data, is highly correlated to ensemble accuracy, and we propose a novel technology for classifier ensemble, learning to diversify, which learns to adaptively combine classifiers by considering both accuracy and diversity. Specifically, our approach, Learning TO Diversify via Weighted Kernels (L2DWK), performs classifier combination by optimizing a direct but simple criterion: maximizing ensemble accuracy and adaptive diversity simultaneously by minimizing a convex loss function. Given a measure formulation, the diversity is calculated with weighted kernels (i.e., the diversity is measured on the component classifiers' outputs which are kernelled and weighted), and the kernel weights are automatically learned. We minimize this loss function by estimating the kernel weights in conjunction with the classifier weights, and propose a self-training algorithm for conducting this convex optimization procedure iteratively. Extensive experiments on a variety of 32 UCI classification benchmark datasets show that the proposed approach consistently outperforms state-of-the-art ensembles such as Bagging, AdaBoost, Random Forests, Gasen, Regularized Selective Ensemble, and Ensemble Pruning via Semi-Definite Programming.

An Ensemble Classifier Based on Attribute Selection and Diversity Measure

Clustering Ensemble with High Diversity Based on Adding Artificial Data

Integration of Classifier Diversity Measures for Feature Selection-Based Classifier Ensemble Reduction.

A New Diversity Measure for Classifier Fusion

Learning to Diversify via Weighted Kernels for Classifier Ensemble

On the Relationships among Various Diversity Measures in Multiple Classifier Systems

Ensemble Algorithms With Diversity Measures For Credit Scoring

Structural Diversity for Decision Tree Ensemble Learning

Ensemble Learning through Diversity Management: Theory, Algorithms, and Applications

Interpretability Diversity for Decision-Tree-Initialized Dendritic Neuron Model Ensemble

Selective ensemble algorithm for gene expression data based on diversity and accuracy of weighted harmonic average measure

Diversity measure for multiple classifier systems

A Novel Diversity Measure of Multiple Classifier Systems Based on Distance of Evidence

An Entropy-Based Diversity Measure for Classifier Combining and Its Application to Face Classifier Ensemble Thinning

Application of adaptive ensemble algorithm based on correctness and diversity

Experimental Study and Comparison of Imbalance Ensemble Classifiers with Dynamic Selection Strategy

Subspace Ensembles for Classification

Construction of Structural Diversity of Ensemble Learning Based on Classification Coding

Generating Diverse and Accurate Classifier Ensembles Using Multi-Objective Optimization

A New Diverse AdaBoost Classifier

Ensemble Classifier Design Based on Perturbation Binary Salp Swarm Algorithm for Classification