Abstract:Deep convolutional neural networks (CNNs) have achieved promising performance in the field of deep learning, but the manual design turns out to be very difficult due to the increasingly complex topologies of CNNs. Recently, neural architecture search (NAS) methods have been proposed to automatically design network architectures, which are superior to handcrafted counterparts. Unfortunately, most current NAS methods suffer from either highly computational complexity of generated architectures or limitations in the flexibility of architecture design. To address above issues, this article proposes an evolutionary neural architecture search (ENAS) method based on improved Transformer and multi-branch ConvNet. The multi-branch block enriches the feature space and enhances the representational capacity of a network by combining paths with different complexities. Since convolution is inherently a local operation, a simple yet powerful “batch-free normalization Transformer Block” (BFNTBlock) is proposed to leverage both local information and long-range feature dependencies. In particular, the design of batch-free normalization (BFN) and batch normalization (BN) mixed in the BFNTBlock blocks the accumulation of estimation shift ascribe to the stack of BN, which has favorable effects for performance improvement. The proposed method achieves remarkable accuracies, 97.24 % and 80.06 % on CIFAR10 and CIFAR100, respectively, with high computational efficiency, i.e. only 1.46 and 1.53 GPU days. To validate the universality of our method in application scenarios, the proposed algorithm is verified on two real-world applications, including the GTSRB and NEU-CLS dataset, and achieves a better performance than common methods.

EDE-NAS: an Eclectic Differential Evolution Approach to Single-Path Neural Architecture Search

A Survey on Evolutionary Neural Architecture Search

Delta-NAS: Difference of Architecture Encoding for Predictor-based Evolutionary Neural Architecture Search

A Gradient-Guided Evolutionary Neural Architecture Search

Evolutionary Neural Architecture Search Combining Multi-Branch ConvNet and Improved Transformer

A Hierarchical Evolution of Neural Architecture Search Method Based on State Transition Algorithm

Neural Architecture Search Based on Evolutionary Algorithms with Fitness Approximation

Evolutionary Neural Network Architecture Search

XC-NAS: A New Cellular Encoding Approach for Neural Architecture Search of Multi-path Convolutional Neural Networks

A Hybrid DE Approach to Designing CNN for Image Classification

Differential Evolution for Architecture Design

Efficient Evolutionary Neural Architecture Search Based on Hybrid Search Space

A Hybrid Differential Evolution Approach to Designing Deep Convolutional Neural Networks for Image Classification

Evolutionary Neural Architecture Search Based on Evaluation Correction and Functional Units

Toward Evolutionary Multitask Convolutional Neural Architecture Search

Split-Level Evolutionary Neural Architecture Search with Elite Weight Inheritance.

Evolution and Efficiency in Neural Architecture Search: Bridging the Gap Between Expert Design and Automated Optimization

A Fast Evolutionary Knowledge Transfer Search for Multiscale Deep Neural Architecture

Automated Design of CNN Architecture Based on Efficient Evolutionary Search

Reinforced Evolutionary Neural Architecture Search

EENA: Efficient Evolution of Neural Architecture.