Abstract:The goal of an ensemble construction with multiple neural networks (NNs) is to achieve better generalization ability in comparison with a single network. Proper diversity among component networks is considered to be an important aspect of neural network ensemble (NNE) construction, in that a failure of one network may be compensated for by other networks. Conventional methods first produce a multiplicity of diverse networks and then combine their decisions for the overall decision of the ensemble. In general, they do not check whether each network is essential for the ensemble, or whether a subset of the networks might perform better. If a particular network performs poorly, it may not be possible for the remaining networks to compensate effectively, resulting in a poorly performing NNE. Although a number of techniques have been investigated in the last few years, no single technique has been discovered that performs well on all possible problems. While a certain solution may outperform alternatives for a subset of problems, the method may perform worse on other problems. This paper discusses Ensembles Fusing Multiple Popular Methods (EMPM), which demonstrate a relatively good performance on all possible problems. EMPM first produces a network pool using several popular methods, and then it selects networks for an ensemble using a proposed forward selection scheme. EMPM has shown better performance with a compact ensemble over conventional methods when tested on a suite of 25 benchmark problems. Experimental analyses have revealed that a heterogeneous network pool of EMPM (using multiple popular methods) is more effective than a pool of any individual conventional method for NN selection. Forward selection has also been found to be a more effective method with a variety of benefits when compared to conventional genetic algorithm-based selection. Index Terms — diversity, generalization ability, neural network ensemble, network selection.

Training of neural network ensemble through progressive interaction

Neural Network Ensemble Training By Sequential Interaction

Progressive interactive training: A sequential neural network ensemble learning method

Adaptive Ensemble Construction Based on Progressive Interactive Training of Neural Networks

Hybrid Ensemble Construction with Selected Neural Networks.

A Constructive Algorithm for Training Neural Network Ensemble

Hybrid neural network ensemble construction combining boosting and negative correlation learning

A New Algorithm for Training Cooperative Neural Network Ensembles

A Constructive Algorithm for Training Cooperative Neural Network Ensembles

A Pruning Algorithm for Training Cooperative Neural Network Ensembles

Concurrent Optimization of Multiple Base Learners in Neural Network Ensembles: an Adaptive Niching Differential Evolution Approach

Bagging and Boosting Negatively Correlated Neural Networks

Compact Ensemble Model of Neural Networks

Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning.

Dynamic Neural Network Ensemble Construction For Classification

Nature-Inspired Neural Network Ensemble Learning

Ensembles of Neural Networks through crossover based pattern generation

A Hybrid Algorithm for Compact Neural Network Ensemble

Design & Implementation of the Parallel-distributed Neural Network Ensemble

Neural Network Ensemble Construction Fusing Multiple Popular Methods

Evolutionary Ensembles with Negative Correlation Learning