Abstract:The goal of an ensemble construction with multiple neural networks (NNs) is to achieve better generalization ability in comparison with a single network. Proper diversity among component networks is considered to be an important aspect of neural network ensemble (NNE) construction, in that a failure of one network may be compensated for by other networks. Conventional methods first produce a multiplicity of diverse networks and then combine their decisions for the overall decision of the ensemble. In general, they do not check whether each network is essential for the ensemble, or whether a subset of the networks might perform better. If a particular network performs poorly, it may not be possible for the remaining networks to compensate effectively, resulting in a poorly performing NNE. Although a number of techniques have been investigated in the last few years, no single technique has been discovered that performs well on all possible problems. While a certain solution may outperform alternatives for a subset of problems, the method may perform worse on other problems. This paper discusses Ensembles Fusing Multiple Popular Methods (EMPM), which demonstrate a relatively good performance on all possible problems. EMPM first produces a network pool using several popular methods, and then it selects networks for an ensemble using a proposed forward selection scheme. EMPM has shown better performance with a compact ensemble over conventional methods when tested on a suite of 25 benchmark problems. Experimental analyses have revealed that a heterogeneous network pool of EMPM (using multiple popular methods) is more effective than a pool of any individual conventional method for NN selection. Forward selection has also been found to be a more effective method with a variety of benefits when compared to conventional genetic algorithm-based selection. Index Terms — diversity, generalization ability, neural network ensemble, network selection.

A Constructive Algorithm for Training Neural Network Ensemble

A Constructive Algorithm for Training Cooperative Neural Network Ensembles

Optimizing Weights by Genetic Algorithm for Neural Network Ensemble

A New Algorithm for Training Cooperative Neural Network Ensembles

Hybrid Ensemble Construction with Selected Neural Networks.

Dynamic Neural Network Ensemble Construction For Classification

Training of neural network ensemble through progressive interaction

Hybrid neural network ensemble construction combining boosting and negative correlation learning

Compact Ensemble Model of Neural Networks

A Neural Network Selection Approach of Constructing Ensemble Based on CS Algorithm

Research and development of neural network ensembles: a survey

Neural Network Ensembles: Theory, Training, and the Importance of Explicit Diversity

Neural Network Ensemble Training By Sequential Interaction

A Pruning Algorithm for Training Cooperative Neural Network Ensembles

The Diversified Ensemble Neural Network.

Method of Incremental Construction of Heterogeneous Neural Network Ensemble with Negative Correlation

Ensembles of Neural Networks through crossover based pattern generation

Concurrent Optimization of Multiple Base Learners in Neural Network Ensembles: an Adaptive Niching Differential Evolution Approach

Neural Subnetwork Ensembles

Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning.

Neural Network Ensemble Construction Fusing Multiple Popular Methods