Abstract:Neural network ensemble is a learning paradigm where a collection of neural networks is trained for the same task. In this paper, the relationship between the generalization ability of the neural network ensemble and the correlation of the individual neural networks constituting the ensemble is analyzed in the context of combining neural regression estimators, which reveals that ensembling a selective subset of trained networks is superior to ensembling all the trained networks in some cases. Based on such recognition, an approach named GASEN is proposed. GASEN trains a number of individual neural networks at first. Then it assigns random weights to the individual networks and employs a genetic algorithm to evolve those weights so that they can characterize to some extent the importance of the individual networks in constituting an ensemble. Finally it selects an optimum subset of individual networks based on the evolved weights to make up the ensemble. Experimental results show that, comparing with a popular ensemble approach, i.e., averaging all, and a theoretically optimum selective ensemble approach, i.e. enumerating, GASEN has preferable performance in generating ensembles with strong generalization ability in relatively small computational cost. This paper also analyzes the working mechanism of GASEN from the view of error-ambiguity decomposition, which reveals that GASEN improves generalization ability mainly through reducing the average generalization error of the individual neural networks constituting the ensemble.

Optimizing Weights by Genetic Algorithm for Neural Network Ensemble

Dynamically Weighted Ensemble Neural Networks with Generalized Regression Neural Network for Solving Regression Problems

Support Vector Machines Ensemble With Optimizing Weights By Genetic Algorithm

Dynamically Weighted Ensemble Neural Networks for Regression Problems

Genetic Ensemble of Extreme Learning Machine

COMBINING REGRESSION ESTIMATORS: GA-BASED SELECTIVE NEURAL NETWORK ENSEMBLE

Genetic Algorithm based Selective Neural Network Ensemble.

Ensembling Neural Networks: Many Could Be Better Than All

Optimizing Ensemble Weights and Hyperparameters of Machine Learning Models for Regression Problems

Using ensemble method to improve the performance of genetic algorithm

MeGA: Merging Multiple Independently Trained Neural Networks Based on Genetic Algorithm

The Diversified Ensemble Neural Network.

Boost Neural Networks by Checkpoints

Improving Deep Neural Network Ensembles Using Reconstruction Error

Effective Neural Network Training with a New Weighting Mechanism-Based Optimization Algorithm.

Optimization Study of BP Neural Network Based on Genetic Algorithm

Ensemble of GA Based Selective Neural Network Ensembles

Group Ensemble: Learning an Ensemble of ConvNets in a single ConvNet

BGADAM: Boosting based Genetic-Evolutionary ADAM for Neural Network Optimization

Selectively ensembling neural classifiers

Optimizing Feed-Forward Neural Networks Using Cascaded Genetic Algorithm