Abstract:Classifier ensemble generally should combine diverse component classifiers. However, it is difficult to give a definitive connection between diversity measure and ensemble accuracy. Given a list of available component classifiers, how to adaptively and diversely ensemble classifiers becomes a big challenge in the literature. In this paper, we argue that diversity, not direct diversity on samples but adaptive diversity with data, is highly correlated to ensemble accuracy, and we propose a novel technology for classifier ensemble, learning to diversify, which learns to adaptively combine classifiers by considering both accuracy and diversity. Specifically, our approach, Learning TO Diversify via Weighted Kernels (L2DWK), performs classifier combination by optimizing a direct but simple criterion: maximizing ensemble accuracy and adaptive diversity simultaneously by minimizing a convex loss function. Given a measure formulation, the diversity is calculated with weighted kernels (i.e., the diversity is measured on the component classifiers' outputs which are kernelled and weighted), and the kernel weights are automatically learned. We minimize this loss function by estimating the kernel weights in conjunction with the classifier weights, and propose a self-training algorithm for conducting this convex optimization procedure iteratively. Extensive experiments on a variety of 32 UCI classification benchmark datasets show that the proposed approach consistently outperforms state-of-the-art ensembles such as Bagging, AdaBoost, Random Forests, Gasen, Regularized Selective Ensemble, and Ensemble Pruning via Semi-Definite Programming.

A New Diversity Measure for Classifier Fusion

An Updatable Classifier Diversity Measure Based on the ER Rule

An Analysis of Diversity Measures

Diversity Measures in Ensemble Learning

On the Relationships among Various Diversity Measures in Multiple Classifier Systems

Integration of Classifier Diversity Measures for Feature Selection-Based Classifier Ensemble Reduction.

An Entropy-Based Diversity Measure for Classifier Combining and Its Application to Face Classifier Ensemble Thinning

Classifier Ensemble with Diversity：Effectiveness Analysis and Ensemble Optimization

A novel ensemble classifier based on multiple diverse classification methods

A Novel Diversity Measure of Multiple Classifier Systems Based on Distance of Evidence

A novel classifier ensemble method with sparsity and diversity

A Novel Diversity Measure Based on Geometrical Relationship

Diversity measure for multiple classifier systems

Learning to Diversify via Weighted Kernels for Classifier Ensemble

The diversity of classifiers and its applications to combination

An Ensemble Classifier Based on Attribute Selection and Diversity Measure

Learning to Diversify for Ensembling of An Amount of Classifiers

Diversity-Based Ensemble with Sample Weight Learning

Convex ensemble learning with sparsity and diversity

Applying Machine Learning Diversity Metrics To Data Fusion In Information Retrieval

A Classification Method for Microarrays Based on Diversity