Abstract:Deep neural networks have played a crucial role in the field of deep learning, achieving significant success in practical applications. The architecture of neural networks is key to their performance. In the past few years, these architectures have been manually designed by experts with rich domain knowledge. Additionally, the optimal neural network architecture can vary depending on specific tasks and data distributions. Neural Architecture Search (NAS) is a class of techniques aimed at automatically searching for and designing neural network architectures according to the given tasks and data. Specifically, evolutionary-computation-based NAS methods are known for their strong global search capability and have aroused widespread interest in recent years. Although evolutionary-computation-based NAS has achieved success in a wide range of research and applications, it still faces bottlenecks in training and evaluating a large number of individuals during optimization. In this study, we first devise a multi-objective evolutionary NAS framework based on a weight-sharing supernet to improve the search efficiency of traditional evolutionary-computation-based NAS. This framework combines the population optimization characteristic of evolutionary algorithms with the weight-sharing ideas in one-shot models. We then design a bi-population MOEA/D algorithm based on the proposed framework to effectively solve the NAS problem. By constructing two sub-populations with different optimization objectives, the algorithm can effectively explore network architectures of various sizes in complex search spaces. An inter-population communication mechanism further enhances the algorithm’s exploratory capability, enabling it to find network architectures with uniform distribution and high diversity. Finally, we conduct performance comparison experiments on image classification datasets of different scales and complexities. Experimental results demonstrate the effectiveness of the proposed multi-objective evolutionary NAS framework and the practicality and transferability of the introduced bi-population MOEA/D-based NAS method compared to existing state-of-the-art NAS methods.

Efficient Network Architecture Search Via Multiobjective Particle Swarm Optimization Based on Decomposition

Particle Swarm Optimization for Compact Neural Architecture Search for Image Classification

Multi-Objective Neural Architecture Search Based on Diverse Structures and Adaptive Recommendation

Neural Architecture Search using Particle Swarm and Ant Colony Optimization

Evolving blocks by segmentation for neural architecture search

Prioritized Architecture Sampling with Monto-Carlo Tree Search

Multi-shot NAS for Discovering Adversarially Robust Convolutional Neural Architectures at Targeted Capacities

Neural Architecture Search Based on Particle Swarm Optimization

Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet

Multi-Objective Neural Architecture Search by Learning Search Space Partitions

M²NAS: Joint Neural Architecture Optimization System with Network Transmission

psoResNet: an Improved PSO-Based Residual Network Search Algorithm

MO-CNN: Multiobjective Optimization of Convolutional Neural Networks for Hyperspectral Image Classification

Particle Swarm Optimisation for Evolving Deep Neural Networks for Image Classification by Evolving and Stacking Transferable Blocks

A Flexible Variable-length Particle Swarm Optimization Approach to Convolutional Neural Network Architecture Design

Neural Architecture Search Based on A Multi-objective Evolutionary Algorithm With Probability Stack

A Pairwise Comparison Relation-assisted Multi-objective Evolutionary Neural Architecture Search Method with Multi-population Mechanism

Multi-Objective Neural Architecture Search for Light-Weight Model

Architecture evolution of convolutional neural network using monarch butterfly optimization

Neural Architecture Search with an Efficient Multiobjective Evolutionary Framework

Automatic Design of Convolutional Neural Network Architectures under Resource Constraints.