Abstract:Neural network ensemble is a learning paradigm where a collection of neural networks is trained for the same task. In this paper, the relationship between the generalization ability of the neural network ensemble and the correlation of the individual neural networks constituting the ensemble is analyzed in the context of combining neural regression estimators, which reveals that ensembling a selective subset of trained networks is superior to ensembling all the trained networks in some cases. Based on such recognition, an approach named GASEN is proposed. GASEN trains a number of individual neural networks at first. Then it assigns random weights to the individual networks and employs a genetic algorithm to evolve those weights so that they can characterize to some extent the importance of the individual networks in constituting an ensemble. Finally it selects an optimum subset of individual networks based on the evolved weights to make up the ensemble. Experimental results show that, comparing with a popular ensemble approach, i.e., averaging all, and a theoretically optimum selective ensemble approach, i.e. enumerating, GASEN has preferable performance in generating ensembles with strong generalization ability in relatively small computational cost. This paper also analyzes the working mechanism of GASEN from the view of error-ambiguity decomposition, which reveals that GASEN improves generalization ability mainly through reducing the average generalization error of the individual neural networks constituting the ensemble.

A New Selective Neural Network Ensemble with Negative Correlation

Optimizing Weights by Genetic Algorithm for Neural Network Ensemble

A hybrid ensemble method with negative correlation learning for regression

Evolutionary Random Neural Ensembles Based on Negative Correlation Learning

Multiobjective Neural Network Ensembles Based on Regularized Negative Correlation Learning.

Genetic Algorithm based Selective Neural Network Ensemble.

Deep Negative Correlation Classification

Regularized Negative Correlation Learning for Neural Network Ensembles

COMBINING REGRESSION ESTIMATORS: GA-BASED SELECTIVE NEURAL NETWORK ENSEMBLE

Novel Selective Support Vector Machine Ensemble Learning Algorithm

Genetic Ensemble of Extreme Learning Machine

Ensemble of GA Based Selective Neural Network Ensembles

Defending Adversarial Examples by Negative Correlation Ensemble

Fast Decorrelated Neural Network Ensembles with Random Weights

A Constructive Algorithm for Training Neural Network Ensemble

Selective Neural Network Ensemble Approach Based on Cultural Algorithm

Negatively correlated ensemble against transfer adversarial attacks

Stochastic Configuration Networks Ensemble with Heterogeneous Features for Large-Scale Data Analytics

Negative Correlation Learning for Classification Ensembles.

Negatively correlated neural network ensemble with multi-population particle swarm optimization

Ensembling Neural Networks: Many Could Be Better Than All