Abstract:Classifier ensemble generally should combine diverse component classifiers. However, it is difficult to give a definitive connection between diversity measure and ensemble accuracy. Given a list of available component classifiers, how to adaptively and diversely ensemble classifiers becomes a big challenge in the literature. In this paper, we argue that diversity, not direct diversity on samples but adaptive diversity with data, is highly correlated to ensemble accuracy, and we propose a novel technology for classifier ensemble, learning to diversify, which learns to adaptively combine classifiers by considering both accuracy and diversity. Specifically, our approach, Learning TO Diversify via Weighted Kernels (L2DWK), performs classifier combination by optimizing a direct but simple criterion: maximizing ensemble accuracy and adaptive diversity simultaneously by minimizing a convex loss function. Given a measure formulation, the diversity is calculated with weighted kernels (i.e., the diversity is measured on the component classifiers' outputs which are kernelled and weighted), and the kernel weights are automatically learned. We minimize this loss function by estimating the kernel weights in conjunction with the classifier weights, and propose a self-training algorithm for conducting this convex optimization procedure iteratively. Extensive experiments on a variety of 32 UCI classification benchmark datasets show that the proposed approach consistently outperforms state-of-the-art ensembles such as Bagging, AdaBoost, Random Forests, Gasen, Regularized Selective Ensemble, and Ensemble Pruning via Semi-Definite Programming.

A Novel Diversity Measure of Multiple Classifier Systems Based on Distance of Evidence

On the Relationships among Various Diversity Measures in Multiple Classifier Systems

A Novel Diversity Measure Based on Geometrical Relationship

A New Diversity Measure for Classifier Fusion

Diversity measure for multiple classifier systems

Integration of Classifier Diversity Measures for Feature Selection-Based Classifier Ensemble Reduction.

A Novel Multi-Source Classification Approach Based on Evidence Combination

Combination of Heterogeneous Multiple Classifiers Based on Evidence Theory

An Ensemble Classifier Based on Attribute Selection and Diversity Measure

Learning to Diversify via Weighted Kernels for Classifier Ensemble

On the Diversity-Performance Relationship for Majority Voting in Classifier Ensembles.

Design of Dynamic Multiple Classifier Systems Based on Belief Functions

Ensemble Algorithms With Diversity Measures For Credit Scoring

Diversity Learning: Introducing the Space-time Scheme to Ensemble Learning

An Entropy-Based Diversity Measure for Classifier Combining and Its Application to Face Classifier Ensemble Thinning

The diversity of classifiers and its applications to combination

Generating Diverse and Accurate Classifier Ensembles Using Multi-Objective Optimization

Combination of Multiple Classifier Using Feature Space Partitioning

Design Of Multiple Classifier Systems Based On Evidential Neural Network

Constructing Classifiers for Imbalanced Data Using Diversity Optimisation

Design of a new classifier simulator