Abstract:Neural architecture search (NAS) is an effective approach for automating the design of deep neural networks. Evolutionary computation (EC) is commonly used in NAS due to its global optimization capability. However, the evaluation phase of architecture candidates in EC-based NAS is compute-intensive, limiting its application for many real-world problems. To overcome this challenge, we propose a novel progressive evaluation strategy for the evaluation phase in convolutional neural network architecture search, in which the number of training epochs of network individuals is progressively increased. In addition, a sub-population preservation strategy is proposed to preserve medium-size and large-size architectures to avoid prematurely discarding networks that may not perform well in the early stages but have the potential to excel with further optimization. Our proposed algorithm reduces the computational cost of the evaluation phase and promotes population diversity and fairness by preserving promising networks based on their distribution. We evaluate the proposed progressive evaluation and sub-population preservation of neural architecture search (PEPNAS) algorithm on the CIFAR10, CIFAR100, and ImageNet benchmark datasets, and compare it with 36 state-of-the-art algorithms, including manually designed networks, reinforcement learning (RL) algorithms, gradient-based algorithms, and other EC-based ones. The experimental results demonstrate that PEPNAS effectively identifies networks with competitive accuracy while also markedly improving the efficiency of the search process. For instance, PEPNAS discovers the architecture on CIFAR10 with a low error rate of 2.38% using only 0.7 GPU days. We directly adopt the searched architecture for the image classification on the CIFAR100 and ImageNet datasets, which achieves the top 1 error rates of 16.46% and 26.25%, respectively. The code is available at https://github.com/chajiajie/PEPNAS.

SELECTING AN ARTIFICIAL NEURAL NETWORK ARCHITECTURE FOR ASSESSING THE STATE OF A GENETIC ALGORITHM POPULATION IN THE PROBLEM OF STRUCTURAL-PARAMETRIC SYNTHESIS OF SIMULATION MODELS OF BUSINESS PROCESSES

Genetic Algorithm Based Recurrent Fuzzy Neural Network Modeling of Chemical Processes

Nonlinear semi-parametric modeling method based on GA-ANN

An automatic selection of optimal recurrent neural network architecture for processes dynamics modelling purposes

Nonlinear Semi-Parametric Modeling Mothed Based on GA-ANN

A neural network approach for population synthesis

Multi-Objective Evolutionary Neural Architecture Search for Recurrent Neural Networks

Can deep neural networks learn process model structure? An assessment framework and analysis

Designing a set of criteria for evaluating artificial neural networks trained with physics-based data to replicate molecular dynamics and other particle method trajectories

Genetic-GNN: Evolutionary architecture search for Graph Neural Networks

Neuroevolution-Based Network Architecture Evolution in Semiconductor Manufacturing

Understanding Neural Architecture Search Techniques

GraphNAS++: Distributed Architecture Search for Graph Neural Networks

A genetic programming approach to the automated design of CNN models for image classification and video shorts creation

GraphPNAS: Learning Distribution of Good Neural Architectures via Deep Graph Generative Models

Auto-GNAS: A Parallel Graph Neural Architecture Search Framework

Genetic Algorithmic Parameter Optimisation of a Recurrent Spiking Neural Network Model

Autonomously and Simultaneously Refining Deep Neural Network Parameters by a Bi-Generative Adversarial Network Aided Genetic Algorithm

Research on safety simulation model and algorithm of dynamic system based on artificial neural network

Neural Architecture Search with Progressive Evaluation and Sub-Population Preservation