Abstract:The goal of an ensemble construction with multiple neural networks (NNs) is to achieve better generalization ability in comparison with a single network. Proper diversity among component networks is considered to be an important aspect of neural network ensemble (NNE) construction, in that a failure of one network may be compensated for by other networks. Conventional methods first produce a multiplicity of diverse networks and then combine their decisions for the overall decision of the ensemble. In general, they do not check whether each network is essential for the ensemble, or whether a subset of the networks might perform better. If a particular network performs poorly, it may not be possible for the remaining networks to compensate effectively, resulting in a poorly performing NNE. Although a number of techniques have been investigated in the last few years, no single technique has been discovered that performs well on all possible problems. While a certain solution may outperform alternatives for a subset of problems, the method may perform worse on other problems. This paper discusses Ensembles Fusing Multiple Popular Methods (EMPM), which demonstrate a relatively good performance on all possible problems. EMPM first produces a network pool using several popular methods, and then it selects networks for an ensemble using a proposed forward selection scheme. EMPM has shown better performance with a compact ensemble over conventional methods when tested on a suite of 25 benchmark problems. Experimental analyses have revealed that a heterogeneous network pool of EMPM (using multiple popular methods) is more effective than a pool of any individual conventional method for NN selection. Forward selection has also been found to be a more effective method with a variety of benefits when compared to conventional genetic algorithm-based selection. Index Terms — diversity, generalization ability, neural network ensemble, network selection.

A Comparative Study of Data Sampling Techniques for Constructing Neural Network Ensembles

Ensembles of Artificial Example Based Neural Networks.

An Approach to Improving Diversity of Neural Network Ensemble

Neural Network Ensemble Construction Fusing Multiple Popular Methods

Ensembles of diverse classifiers using synthetic training data

Hybrid Ensemble Construction with Selected Neural Networks.

Neural Network Ensembles: Theory, Training, and the Importance of Explicit Diversity

Ensembles of Neural Networks through crossover based pattern generation

Ensembling Neural Networks: Many Could Be Better Than All

A New Ensemble Learning Algorithm Based on Improved K-Means for Training Neural Network Ensembles

Neural Network Ensemble Training By Sequential Interaction

The Diversified Ensemble Neural Network.

Diversity-Based Ensemble with Sample Weight Learning

Why m heads are better than one: Training a diverse ensemble of deep networks

Ensembles Of Neural Networks Based On The Alteration Of Input Feature Values

Ensemble Algorithm Of Neural Networks And Its Application

A Neural Network Selection Approach of Constructing Ensemble Based on CS Algorithm

Efficient Diversity-Driven Ensemble for Deep Neural Networks

Bagging and Boosting Negatively Correlated Neural Networks

A New Algorithm for Training Cooperative Neural Network Ensembles

Trade-Off Between Diversity and Accuracy in Ensemble Generation