Abstract:The goal of an ensemble construction with multiple neural networks (NNs) is to achieve better generalization ability in comparison with a single network. Proper diversity among component networks is considered to be an important aspect of neural network ensemble (NNE) construction, in that a failure of one network may be compensated for by other networks. Conventional methods first produce a multiplicity of diverse networks and then combine their decisions for the overall decision of the ensemble. In general, they do not check whether each network is essential for the ensemble, or whether a subset of the networks might perform better. If a particular network performs poorly, it may not be possible for the remaining networks to compensate effectively, resulting in a poorly performing NNE. Although a number of techniques have been investigated in the last few years, no single technique has been discovered that performs well on all possible problems. While a certain solution may outperform alternatives for a subset of problems, the method may perform worse on other problems. This paper discusses Ensembles Fusing Multiple Popular Methods (EMPM), which demonstrate a relatively good performance on all possible problems. EMPM first produces a network pool using several popular methods, and then it selects networks for an ensemble using a proposed forward selection scheme. EMPM has shown better performance with a compact ensemble over conventional methods when tested on a suite of 25 benchmark problems. Experimental analyses have revealed that a heterogeneous network pool of EMPM (using multiple popular methods) is more effective than a pool of any individual conventional method for NN selection. Forward selection has also been found to be a more effective method with a variety of benefits when compared to conventional genetic algorithm-based selection. Index Terms — diversity, generalization ability, neural network ensemble, network selection.

Ensembles of Neural Networks through crossover based pattern generation

Optimizing Weights by Genetic Algorithm for Neural Network Ensemble

Ensembles of Artificial Example Based Neural Networks.

Ensembles Of Neural Networks Based On The Alteration Of Input Feature Values

Neural Network Ensemble Construction Fusing Multiple Popular Methods

Genetic Ensemble of Extreme Learning Machine

Hybrid Ensemble Construction with Selected Neural Networks.

Concurrent Optimization of Multiple Base Learners in Neural Network Ensembles: an Adaptive Niching Differential Evolution Approach

A New Algorithm for Training Cooperative Neural Network Ensembles

Towards Designing Neural Network Ensembles by Evolution

Evolving Artificial Neural Network Ensembles

Ensembling Neural Networks: Many Could Be Better Than All

A Constructive Algorithm for Training Neural Network Ensemble

Neural Network Ensemble Training By Sequential Interaction

Neural Network Ensembles: Theory, Training, and the Importance of Explicit Diversity

Coevolutionary Learning of Neural Network Ensemble for Complex Classification Tasks

Nature-Inspired Neural Network Ensemble Learning

Ensemble Algorithm Of Neural Networks And Its Application

A Comparative Study of Data Sampling Techniques for Constructing Neural Network Ensembles

A Constructive Algorithm for Training Cooperative Neural Network Ensembles

Ensembles of diverse classifiers using synthetic training data