Abstract:As an advanced artificial intelligence technique for solving learning problems, deep learning (DL) has achieved great success in many real-world applications and attracted increasing attention in recent years. However, as the performance of DL depends on many factors such as the architecture and hyperparameters, how to optimize DL has become a hot research topic in the field of DL and artificial intelligence. Evolutionary computation (EC), including evolutionary algorithm and swarm intelligence, is a kind of efficient and intelligent optimization methodology inspired by the mechanisms of biological evolution and behaviors of swarm organisms. Therefore, a large number of researches have proposed EC algorithms to optimize DL, so called evolutionary deep learning (EDL), which have obtained promising results. Given the great progress and rapid development of EDL in recent years, it is quite necessary to review these developments in order to summarize previous research experiences and knowledge, as well as provide references to benefit the development of more researches and applications. For this aim, this paper categorizes existing works in a two-level taxonomy. The higher level includes four categories based on when the EC can be adopted in optimizing the DL, which are the four procedures of the whole DL lifetime, including data processing, model search, model training, and model evaluation and utilization. In the lower level, related works in each category are further classified according to the functionality and the aim of using EC in the corresponding DL procedure, i.e., why using EC in this DL procedure. As a result, the taxonomy can clearly show how an EC algorithm can be used to optimize and improve DL. Moreover, this survey also discusses the potential research directions to provide the prospect of EDL in the future.

A Survey on Evolutionary Construction of Deep Neural Networks.

A Survey of Advances in Evolutionary Neural Architecture Search

Neuroevolution in Deep Neural Networks: Current Trends and Future Challenges

A Survey on Evolutionary Neural Architecture Search

DECN: Automated Evolutionary Algorithms Via Evolution Inspired Deep Convolution Network

A survey: evolutionary deep learning

DECN: Evolution Inspired Deep Convolution Network for Black-box Optimization

Survey on Evolutionary Deep Learning: Principles, Algorithms, Applications and Open Issues

Guest Editorial Evolutionary Neural Architecture Search

A Gradient-Guided Evolutionary Approach to Training Deep Neural Networks

Evolutionary neural networks for deep learning: a review

Evolutionary Neural Network Architecture Search

A Self-adaptive neuroevolution approach to constructing Deep Neural Network architectures across different types

Evolutionary Shallowing Deep Neural Networks at Block Levels

Evolutionary deep learning: A survey

Asynchronous Evolution of Deep Neural Network Architectures

Evolutional Neural Architecture Search for Optimization of Spatiotemporal Brain Network Decomposition.

Is it conceivable that neurogenesis, neural Darwinism, and species evolution could all serve as inspiration for the creation of evolutionary deep neural networks?

A Survey on Neural Architecture Search

When Neural Network Computation Meets Evolutionary Computation: A Survey

Automated Design of CNN Architecture Based on Efficient Evolutionary Search