Abstract:Classifier ensemble generally should combine diverse component classifiers. However, it is difficult to give a definitive connection between diversity measure and ensemble accuracy. Given a list of available component classifiers, how to adaptively and diversely ensemble classifiers becomes a big challenge in the literature. In this paper, we argue that diversity, not direct diversity on samples but adaptive diversity with data, is highly correlated to ensemble accuracy, and we propose a novel technology for classifier ensemble, learning to diversify, which learns to adaptively combine classifiers by considering both accuracy and diversity. Specifically, our approach, Learning TO Diversify via Weighted Kernels (L2DWK), performs classifier combination by optimizing a direct but simple criterion: maximizing ensemble accuracy and adaptive diversity simultaneously by minimizing a convex loss function. Given a measure formulation, the diversity is calculated with weighted kernels (i.e., the diversity is measured on the component classifiers' outputs which are kernelled and weighted), and the kernel weights are automatically learned. We minimize this loss function by estimating the kernel weights in conjunction with the classifier weights, and propose a self-training algorithm for conducting this convex optimization procedure iteratively. Extensive experiments on a variety of 32 UCI classification benchmark datasets show that the proposed approach consistently outperforms state-of-the-art ensembles such as Bagging, AdaBoost, Random Forests, Gasen, Regularized Selective Ensemble, and Ensemble Pruning via Semi-Definite Programming.

Negative Correlation Learning for Classification Ensembles.

Regularized Negative Correlation Learning for Neural Network Ensembles

A hybrid ensemble method with negative correlation learning for regression

Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning.

Semisupervised Negative Correlation Learning

Evolutionary Random Neural Ensembles Based on Negative Correlation Learning

Learning and Exploiting Interclass Visual Correlations for Medical Image Classification

Multi-class AdaBoost ELM

Debiased Graph Contrastive Learning.

Nonlinear Regression via Deep Negative Correlation Learning

Negatively correlated ensemble against transfer adversarial attacks

Algorithms for Exploiting Negative Correlation

Learning to Diversify via Weighted Kernels for Classifier Ensemble

Dynamically building diversified classifier pruning ensembles via canonical correlation analysis

Generalized Ambiguity Decompositions for Classification with Applications in Active Learning and Unsupervised Ensemble Pruning.

A New Diverse AdaBoost Classifier

Non-negative Contrastive Learning

Amended Cross Entropy Cost: Framework For Explicit Diversity Encouragement

Neighbor cleaning learning based cost‐sensitive ensemble learning approach for software defect prediction

A Novel Approach to Fully Representing the Diversity in Conditional Dependencies for Learning Bayesian Network Classifier

DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation