Abstract:Deep neural networks have played a crucial role in the field of deep learning, achieving significant success in practical applications. The architecture of neural networks is key to their performance. In the past few years, these architectures have been manually designed by experts with rich domain knowledge. Additionally, the optimal neural network architecture can vary depending on specific tasks and data distributions. Neural Architecture Search (NAS) is a class of techniques aimed at automatically searching for and designing neural network architectures according to the given tasks and data. Specifically, evolutionary-computation-based NAS methods are known for their strong global search capability and have aroused widespread interest in recent years. Although evolutionary-computation-based NAS has achieved success in a wide range of research and applications, it still faces bottlenecks in training and evaluating a large number of individuals during optimization. In this study, we first devise a multi-objective evolutionary NAS framework based on a weight-sharing supernet to improve the search efficiency of traditional evolutionary-computation-based NAS. This framework combines the population optimization characteristic of evolutionary algorithms with the weight-sharing ideas in one-shot models. We then design a bi-population MOEA/D algorithm based on the proposed framework to effectively solve the NAS problem. By constructing two sub-populations with different optimization objectives, the algorithm can effectively explore network architectures of various sizes in complex search spaces. An inter-population communication mechanism further enhances the algorithm’s exploratory capability, enabling it to find network architectures with uniform distribution and high diversity. Finally, we conduct performance comparison experiments on image classification datasets of different scales and complexities. Experimental results demonstrate the effectiveness of the proposed multi-objective evolutionary NAS framework and the practicality and transferability of the introduced bi-population MOEA/D-based NAS method compared to existing state-of-the-art NAS methods.

Objective-hierarchy Based Large-Scale Evolutionary Algorithm for Improving Joint Sparsity-Compression of Neural Network.

Evolving Efficient Genetic Encoding for Deep Spiking Neural Networks

An improved large-scale sparse multi-objective evolutionary algorithm using unsupervised neural network

Neural Network Compression by Joint Sparsity Promotion and Redundancy Reduction

Compression of deep neural networks: bridging the gap between conventional-based pruning and evolutionary approach

Solving Large-Scale Multiobjective Optimization Problems With Sparse Optimal Solutions via Unsupervised Neural Networks

A Knee-Guided Evolutionary Algorithm for Compressing Deep Neural Networks

Multi-Objective Compression for CNNs via Evolutionary Algorithm

Efficient Neural Network Compression Inspired by Compressive Sensing.

Multiobjective Visual Evolutionary Neural Network and Related Convolutional Neural Network Optimization

Multi-Objective Evolutionary Neural Architecture Search with Weight-Sharing Supernet

Reliable Network Search Based on Evolutionary Algorithm

Joint design and compression of convolutional neural networks as a Bi-level optimization problem

Sparse Evolutionary Deep Learning with over One Million Artificial Neurons on Commodity Hardware

CCSRP: Robust Pruning of Spiking Neural Networks through Cooperative Coevolution

Neural Network Compression Via Sparse Optimization

An Aggressive Genetic Programming Approach for Searching Neural Network Structure Under Computational Constraints

Optimization of Neural Network Based on Cooperative Co-Evolutionary Algorithm

A Preference-Based Evolutionary Biobjective Approach for Learning Large-Scale Fuzzy Cognitive Maps: an Application to Gene Regulatory Network Reconstruction.

An evolutionary algorithm based on rank-1 approximation for sparse large-scale multi-objective problems

Simultaneous Weight and Architecture Optimization for Neural Networks