Abstract:Classifier ensemble generally should combine diverse component classifiers. However, it is difficult to give a definitive connection between diversity measure and ensemble accuracy. Given a list of available component classifiers, how to adaptively and diversely ensemble classifiers becomes a big challenge in the literature. In this paper, we argue that diversity, not direct diversity on samples but adaptive diversity with data, is highly correlated to ensemble accuracy, and we propose a novel technology for classifier ensemble, learning to diversify, which learns to adaptively combine classifiers by considering both accuracy and diversity. Specifically, our approach, Learning TO Diversify via Weighted Kernels (L2DWK), performs classifier combination by optimizing a direct but simple criterion: maximizing ensemble accuracy and adaptive diversity simultaneously by minimizing a convex loss function. Given a measure formulation, the diversity is calculated with weighted kernels (i.e., the diversity is measured on the component classifiers' outputs which are kernelled and weighted), and the kernel weights are automatically learned. We minimize this loss function by estimating the kernel weights in conjunction with the classifier weights, and propose a self-training algorithm for conducting this convex optimization procedure iteratively. Extensive experiments on a variety of 32 UCI classification benchmark datasets show that the proposed approach consistently outperforms state-of-the-art ensembles such as Bagging, AdaBoost, Random Forests, Gasen, Regularized Selective Ensemble, and Ensemble Pruning via Semi-Definite Programming.

Ensemble Learning through Diversity Management: Theory, Algorithms, and Applications

Clustering Ensemble with High Diversity Based on Adding Artificial Data

Classifier Ensemble with Diversity：Effectiveness Analysis and Ensemble Optimization

Classifier Ensemble Using A Heuristic Learning With Sparsity And Diversity

Multi-objective Ensemble Generation

A novel classifier ensemble method with sparsity and diversity

On Diversity and Accuracy of Homogeneous and Heterogeneous Ensembles

Trade-Off Between Diversity and Accuracy in Ensemble Generation

A Unified Theory of Diversity in Ensemble Learning

Diversity Analysis on Imbalanced Data Sets by Using Ensemble Models

Investigation on Diversity in Homogeneous and Heterogeneous Ensembles

Ensemble Methods: Foundations and Algorithms

Neural Network Ensembles: Theory, Training, and the Importance of Explicit Diversity

Diversity Learning: Introducing the Space-time Scheme to Ensemble Learning

Diversity Measures in Ensemble Learning

Learning to Diversify via Weighted Kernels for Classifier Ensemble

Diversity Creation Methods: a Survey and Categorisation.

Maximizing Diversity by Transformed Ensemble Learning

Developing parsimonious ensembles using ensemble diversity within a reinforcement learning framework

Testing of Diversity Strategy and Ensemble Strategy in SVM-Based Multiagent Ensemble Learning

Convex ensemble learning with sparsity and diversity