Abstract:We consider the classifier ensemble problem in this paper. Due to its superior performance to individual classifiers, class ensemble has been intensively studied in the literature. Generally speaking, there are two prevalent research directions on this, i.e., to diversely generate classifier components, and to sparsely combine multiple classifiers. While most current approaches are emphasized on either sparsity or diversity only, we investigate the classifier ensemble by learning both sparsity and diversity simultaneously. We manage to formulate the classifier ensemble problem with the sparsity or/and diversity learning in a general framework. In particular, the classifier ensemble with sparsity and diversity can be represented as a mathematical optimization problem. We then propose a heuristic algorithm, capable of obtaining ensemble classifiers with consideration of both sparsity and diversity. We exploit the genetic algorithm, and optimize sparsity and diversity for classifier selection and combination heuristically and iteratively. As one major contribution, we introduce the concept of the diversity contribution ability so as to select proper classifier components and evolve classifier weights eventually. Finally, we compare our proposed novel method with other conventional classifier ensemble methods such as Bagging, least squares combination, sparsity learning, and AdaBoost, extensively on UCI benchmark data sets and the Pascal Large Scale Learning Challenge 2008 webspam data. The experimental results confirm that our approach leads to better performance in many aspects.

Multi-information Ensemble Diversity

Clustering Ensemble with High Diversity Based on Adding Artificial Data

Approximate Multi-Information Diversity*

Diversity Measures in Ensemble Learning

Diversity Learning: Introducing the Space-time Scheme to Ensemble Learning

Ensemble Learning through Diversity Management: Theory, Algorithms, and Applications

Multi-objective Ensemble Generation

On Diversity and Accuracy of Homogeneous and Heterogeneous Ensembles

Classifier Ensemble Using A Heuristic Learning With Sparsity And Diversity

A Unified Theory of Diversity in Ensemble Learning

Investigation on Diversity in Homogeneous and Heterogeneous Ensembles

Classifier Ensemble with Diversity：Effectiveness Analysis and Ensemble Optimization

A New Diversity Measure for Classifier Fusion

Integration of Classifier Diversity Measures for Feature Selection-Based Classifier Ensemble Reduction.

An Analysis of Diversity Measures

Optimising Ensemble Combination Based on Maximisation of Diversity

Maximizing Diversity by Transformed Ensemble Learning

Testing of Diversity Strategy and Ensemble Strategy in SVM-Based Multiagent Ensemble Learning

A novel classifier ensemble method with sparsity and diversity

Generating Diverse and Accurate Classifier Ensembles Using Multi-Objective Optimization

Interpretability Diversity for Decision-Tree-Initialized Dendritic Neuron Model Ensemble