Abstract:Neural architecture search (NAS) is an effective approach for automating the design of deep neural networks. Evolutionary computation (EC) is commonly used in NAS due to its global optimization capability. However, the evaluation phase of architecture candidates in EC-based NAS is compute-intensive, limiting its application for many real-world problems. To overcome this challenge, we propose a novel progressive evaluation strategy for the evaluation phase in convolutional neural network architecture search, in which the number of training epochs of network individuals is progressively increased. In addition, a sub-population preservation strategy is proposed to preserve medium-size and large-size architectures to avoid prematurely discarding networks that may not perform well in the early stages but have the potential to excel with further optimization. Our proposed algorithm reduces the computational cost of the evaluation phase and promotes population diversity and fairness by preserving promising networks based on their distribution. We evaluate the proposed progressive evaluation and sub-population preservation of neural architecture search (PEPNAS) algorithm on the CIFAR10, CIFAR100, and ImageNet benchmark datasets, and compare it with 36 state-of-the-art algorithms, including manually designed networks, reinforcement learning (RL) algorithms, gradient-based algorithms, and other EC-based ones. The experimental results demonstrate that PEPNAS effectively identifies networks with competitive accuracy while also markedly improving the efficiency of the search process. For instance, PEPNAS discovers the architecture on CIFAR10 with a low error rate of 2.38% using only 0.7 GPU days. We directly adopt the searched architecture for the image classification on the CIFAR100 and ImageNet datasets, which achieves the top 1 error rates of 16.46% and 26.25%, respectively. The code is available at https://github.com/chajiajie/PEPNAS.

Neural Architecture Search Based on Particle Swarm Optimization

Neural Architecture Search using Particle Swarm and Ant Colony Optimization

Particle Swarm Optimization for Compact Neural Architecture Search for Image Classification

psoResNet: an Improved PSO-Based Residual Network Search Algorithm

Efficient Network Architecture Search Via Multiobjective Particle Swarm Optimization Based on Decomposition

Neural Architecture Search Based on Improved Brain Storm Optimization Algorithm.

Neural Architecture Search with Progressive Evaluation and Sub-Population Preservation

Evolving Deep Architecture Generation with Residual Connections for Image Classification Using Particle Swarm Optimization

A Flexible Variable-length Particle Swarm Optimization Approach to Convolutional Neural Network Architecture Design

Automating Neural Architecture Design without Search

Evolving blocks by segmentation for neural architecture search

Neural architecture optimization

Dropout Topology-Assisted Bidirectional Learning Particle Swarm Optimization for Neural Architecture Search.

Particle swarm optimization of deep neural networks architectures for image classification

Neural Architecture Search Based on A Multi-objective Evolutionary Algorithm With Probability Stack

Multi-Objective Neural Architecture Search Based on Diverse Structures and Adaptive Recommendation

Scale Adaptive Fitness Evaluation‐based Particle Swarm Optimisation for Hyperparameter and Architecture Optimisation in Neural Networks and Deep Learning

Searching for A Robust Neural Architecture in Four GPU Hours

You Only Search Once: Single Shot Neural Architecture Search via Direct Sparse Optimization

A neural architecture generator for efficient search space

Neural Architecture Search Via Reference Point Based Multi‐objective Evolutionary Algorithm