Abstract:Due to the dominant position of deep learning (mostly deep neural networks) in various artificial intelligence applications, recently, ensemble learning based on deep neural networks (ensemble deep learning) has shown significant performances in improving the generalization of learning system. However, since modern deep neural networks usually have millions to billions of parameters, the time and space overheads for training multiple base deep learners and testing with the ensemble deep learner are far greater than that of traditional ensemble learning. Though several algorithms of fast ensemble deep learning have been proposed to promote the deployment of ensemble deep learning in some applications, further advances still need to be made for many applications in specific fields, where the developing time and computing resources are usually restricted or the data to be processed is of large dimensionality. An urgent problem needs to be solved is how to take the significant advantages of ensemble deep learning while reduce the required expenses so that many more applications in specific fields can benefit from it. For the alleviation of this problem, it is essential to know about how ensemble learning has developed under the era of deep learning. Thus, in this article, we present discussions focusing on data analyses of published works, methodologies, recent advances and unattainability of traditional ensemble learning and ensemble deep learning. We hope this article will be helpful to realize the intrinsic problems and technical challenges faced by future developments of ensemble learning under the era of deep learning.

Research and development of neural network ensembles: a survey

Optimizing Weights by Genetic Algorithm for Neural Network Ensemble

A Constructive Algorithm for Training Neural Network Ensemble

Neural Subnetwork Ensembles

A Survey on ensemble learning under the era of deep learning

Genetic Algorithm based Selective Neural Network Ensemble.

Neural Network Ensembles: Theory, Training, and the Importance of Explicit Diversity

Ensembling Neural Networks: Many Could Be Better Than All

Ensemble of GA Based Selective Neural Network Ensembles

Ensemble Neural Networks (ENN): A gradient-free stochastic method

The Diversified Ensemble Neural Network.

SAE: Single Architecture Ensemble Neural Networks

A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects

Ensemble Multifeatured Deep Learning Models and Applications: A Survey

A Survey on Neural Question Generation: Methods, Applications, and Prospects

Ensemble deep learning: A review

A DNN-Ensemble Method for Error Reduction and Training Data Selection in DNN Based Modeling

NeuE: Automated Neural Network Ensembles for Edge Intelligence

“In-Network Ensemble”: Deep Ensemble Learning with Diversified Knowledge Distillation

Evolutionary neural networks for deep learning: a review

COMBINING REGRESSION ESTIMATORS: GA-BASED SELECTIVE NEURAL NETWORK ENSEMBLE