Abstract:Ensemble learning is a process by which multiple base learners are strategically generated and combined into one composite learner. There are two features that are essential to an ensemble's performance, the individual accuracies of the component learners and the overall diversity in the ensemble. The right balance of learner accuracy and ensemble diversity can improve the performance of machine learning tasks on benchmark and real-world data sets, and recent theoretical and practical work has demonstrated the subtle trade-off between accuracy and diversity in an ensemble. In this paper, we extend the extant literature by providing a deeper theoretical understanding for assessing and improving the optimality of any given ensemble, including random forests and deep neural network ensembles. We also propose a training algorithm for neural network ensembles and demonstrate that our approach provides improved performance when compared to both state-of-the-art individual learners and ensembles of state-of-the-art learners trained using standard loss functions. Our key insight is that it is better to explicitly encourage diversity in an ensemble, rather than merely allowing diversity to occur by happenstance, and that rigorous theoretical bounds on the trade-off between diversity and learner accuracy allow one to know when an optimal arrangement has been achieved.

A New Algorithm for Training Cooperative Neural Network Ensembles

A Constructive Algorithm for Training Cooperative Neural Network Ensembles

A Pruning Algorithm for Training Cooperative Neural Network Ensembles

A Constructive Algorithm for Training Neural Network Ensemble

Bagging and Boosting Negatively Correlated Neural Networks

Hybrid Ensemble Construction with Selected Neural Networks.

Exploring constructive algorithms with stopping criteria to produce accurate and diverse individual neural networks in an ensemble

A cooperative ensemble learning system

Simultaneous Training of Negatively Correlated Neural Networks in an Ensemble.

Neural Network Ensembles: Theory, Training, and the Importance of Explicit Diversity

Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning.

Training of neural network ensemble through progressive interaction

Simultaneous training of negatively correlated neural networks in an ensemble

Towards Designing Neural Network Ensembles by Evolution

Ensembles of Neural Networks through crossover based pattern generation

A Hybrid Algorithm for Compact Neural Network Ensemble

Evolutionary Ensembles with Negative Correlation Learning

Hybrid neural network ensemble construction combining boosting and negative correlation learning

Compact Ensemble Model of Neural Networks

Evolving, training and designing neural network ensembles

Evolving Artificial Neural Network Ensembles