Abstract:As an advanced artificial intelligence technique for solving learning problems, deep learning (DL) has achieved great success in many real-world applications and attracted increasing attention in recent years. However, as the performance of DL depends on many factors such as the architecture and hyperparameters, how to optimize DL has become a hot research topic in the field of DL and artificial intelligence. Evolutionary computation (EC), including evolutionary algorithm and swarm intelligence, is a kind of efficient and intelligent optimization methodology inspired by the mechanisms of biological evolution and behaviors of swarm organisms. Therefore, a large number of researches have proposed EC algorithms to optimize DL, so called evolutionary deep learning (EDL), which have obtained promising results. Given the great progress and rapid development of EDL in recent years, it is quite necessary to review these developments in order to summarize previous research experiences and knowledge, as well as provide references to benefit the development of more researches and applications. For this aim, this paper categorizes existing works in a two-level taxonomy. The higher level includes four categories based on when the EC can be adopted in optimizing the DL, which are the four procedures of the whole DL lifetime, including data processing, model search, model training, and model evaluation and utilization. In the lower level, related works in each category are further classified according to the functionality and the aim of using EC in the corresponding DL procedure, i.e., why using EC in this DL procedure. As a result, the taxonomy can clearly show how an EC algorithm can be used to optimize and improve DL. Moreover, this survey also discusses the potential research directions to provide the prospect of EDL in the future.

Evolutionary neural networks for deep learning: a review

A Review of Evolutionary Artificial Neural Networks

Evolutionary artificial neural networks: a review

A Survey of Advances in Evolutionary Neural Architecture Search

Ensemble Structure of Evolutionary Artificial Neural Networks

Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

A Survey on Evolutionary Neural Architecture Search

Evolutionary Artificial Neural Networks That Learn and Generalise Well

Evolutionary Neural Network Architecture Search

A Survey on Evolutionary Construction of Deep Neural Networks.

Evolutionary Neural Architecture Search and Applications [guest Editorial]

Nature-Inspired Neural Network Ensemble Learning

A Gradient-Guided Evolutionary Approach to Training Deep Neural Networks

A New Evolutionary System for Evolving Artificial Neural Networks

Evolutionary Architecture Search Via Adaptive Parameter Control and Gene Potential Contribution

A Self-adaptive neuroevolution approach to constructing Deep Neural Network architectures across different types

A survey: evolutionary deep learning

A New Evolutionary Neural Network And Its Application For The Extraction Of Vegetation Anomalies

Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications and Open Issues

Evolutionary deep learning: A survey

Evolutionary Neural Architecture Search Based on Evaluation Correction and Functional Units