Abstract:The goal of an ensemble construction with multiple neural networks (NNs) is to achieve better generalization ability in comparison with a single network. Proper diversity among component networks is considered to be an important aspect of neural network ensemble (NNE) construction, in that a failure of one network may be compensated for by other networks. Conventional methods first produce a multiplicity of diverse networks and then combine their decisions for the overall decision of the ensemble. In general, they do not check whether each network is essential for the ensemble, or whether a subset of the networks might perform better. If a particular network performs poorly, it may not be possible for the remaining networks to compensate effectively, resulting in a poorly performing NNE. Although a number of techniques have been investigated in the last few years, no single technique has been discovered that performs well on all possible problems. While a certain solution may outperform alternatives for a subset of problems, the method may perform worse on other problems. This paper discusses Ensembles Fusing Multiple Popular Methods (EMPM), which demonstrate a relatively good performance on all possible problems. EMPM first produces a network pool using several popular methods, and then it selects networks for an ensemble using a proposed forward selection scheme. EMPM has shown better performance with a compact ensemble over conventional methods when tested on a suite of 25 benchmark problems. Experimental analyses have revealed that a heterogeneous network pool of EMPM (using multiple popular methods) is more effective than a pool of any individual conventional method for NN selection. Forward selection has also been found to be a more effective method with a variety of benefits when compared to conventional genetic algorithm-based selection. Index Terms — diversity, generalization ability, neural network ensemble, network selection.

Ensembles of Artificial Example Based Neural Networks.

A Comparative Study of Data Sampling Techniques for Constructing Neural Network Ensembles

Neural Network Ensemble Construction Fusing Multiple Popular Methods

Neural Network Ensembles: Theory, Training, and the Importance of Explicit Diversity

Ensembles of Neural Networks through crossover based pattern generation

An Approach to Improving Diversity of Neural Network Ensemble

The Diversified Ensemble Neural Network.

Optimizing Weights by Genetic Algorithm for Neural Network Ensemble

Ensembles of diverse classifiers using synthetic training data

Efficient Diversity-Driven Ensemble for Deep Neural Networks

Deep Neural Network Ensembles against Deception: Ensemble Diversity, Accuracy and Robustness

Hybrid Ensemble Construction with Selected Neural Networks.

Why m heads are better than one: Training a diverse ensemble of deep networks

Ensembling Neural Networks: Many Could Be Better Than All

Ensembles Of Neural Networks Based On The Alteration Of Input Feature Values

Inverse Adversarial Diversity Learning for Network Ensemble

Developing parsimonious ensembles using ensemble diversity within a reinforcement learning framework

Sequential Bayesian Neural Subnetwork Ensembles

Ensembles of Multi-Instance Neural Networks

A New Algorithm for Training Cooperative Neural Network Ensembles

Dynamic Post-Hoc Neural Ensemblers