Evolutionary Neural Network Architecture Search

Zeqiong Lv,Xiaotian Song,Yue Feng,Yuwei Ou,Yanan Sun,Mengjie Zhang

2023-01-01

Abstract:Deep Neural Networks (DNNs) have been remarkably successful in numerous scenarios of machine learning. However, the typical design for DNN architectures is manual, which highly relies on the domain knowledge and experience of neural networks. Neural architecture search (NAS) methods are often considered an effective way to achieve automated design of DNN architectures. There are three approaches to realizing NAS: reinforcement learningReinforcement Learning approaches, gradient-based approaches, and evolutionary computation approaches. Among them, evolutionary computation-based NASNeural Architecture Search (NAS) (ENAS) has received much attention. This chapter will detail ENAS in terms of four aspects. First, we will present an overall introduction to NAS and the commonly used approaches to NASNeural Architecture Search (NAS). Following that, we will introduce the core components of ENAS and discuss the details of how to design an ENAS algorithm with a focus on search space, search strategy, and performance evaluation of the ENAS algorithm. Moreover, detailed implementations of these components will be presented to help readers implement an ENAS algorithm step by step. We will discuss state-of-the-art ENAS methods with the three core components. Finally, we will provide five major challenges and identify corresponding future directions.