Abstract:With the rise of various smart electronics and mobile/edge devices, many existing high-accuracy convolutional neural network (CNN) models are difficult to be applied in practice due to the limited resources, such as memory capacity, power consumption, and spectral efficiency. In order to meet these constraints, researchers have carefully designed some lightweight networks. Meanwhile, to reduce the reliance on manual design on expert experience, some researchers also work to improve neural architecture search (NAS) algorithms to automatically design small networks, exploiting the multiobjective approaches that consider both accuracy and other important goals during optimization. However, simply searching for smaller network models is not consistent with the current research belief of “the deeper the better” and may affect the effectiveness of the model and thus waste the limited resources available. In this article, we propose an automatic method for designing CNNs architectures under constraint handling, which can search for optimal network models meeting the preset constraint. Specifically, an adaptive penalty algorithm is used for fitness evaluation, and a selective repair operation is developed for infeasible individuals to search for feasible CNN architectures. As a case study, we set the complexity (the number of parameters) as a resource constraint and perform multiple experiments on CIFAR-10 and CIFAR-100, to demonstrate the effectiveness of the proposed method. In addition, the proposed algorithm is compared with a state-of-the-art algorithm, NSGA-Net, and several manual-designed models. The experimental results show that the proposed algorithm can successfully solve the problem of the uncertain size of the optimal CNN model under the random search strategy, and the automatically designed CNN model can satisfy the predefined resource constraint while achieving better accuracy.

Evolutionary Structure Optimization of Convolutional Neural Networks for Deployment on Resource Limited Systems.

An Aggressive Genetic Programming Approach for Searching Neural Network Structure Under Computational Constraints

Evolution under Length Constraints for CNN Architecture design

Multi-Objective Evolutionary Search of Compact Convolutional Neural Networks with Training-Free Estimation

Surrogate-Assisted Evolution of Convolutional Neural Networks by Collaboratively Optimizing the Basic Blocks and Topologies

Multiobjective Evolutionary Design of Deep Convolutional Neural Networks for Image Classification

Convolutional Neural Network Optimization Using Modified NSGA-II

A Fast Evolutionary Learning to Optimize CNNInspec keywordsOther keywordsKey words

Multiobjective Visual Evolutionary Neural Network and Related Convolutional Neural Network Optimization

A Fast Evolutionary Learning to Optimize CNN

Evolutionary Convolutional Neural Network Optimization with Cross-Tasks Transfer Strategy

Automatic Design of Convolutional Neural Network Architectures under Resource Constraints.

Towards Evolutionary Compression.

Optimizing Convolutional Neural Network Architecture

Two Novel Performance Improvements for Evolving CNN Topologies

Evolving Convolutional Neural Networks by Symbiotic Organisms Search Algorithm for Image Classification

Architecture evolution of convolutional neural network using monarch butterfly optimization

Topological optimization of the DenseNet with pretrained-weights inheritance and genetic channel selection

An Efficient Evolutionary Deep Learning Framework Based on Multi-source Transfer Learning to Evolve Deep Convolutional Neural Networks

A Graph-Based Encoding for Evolutionary Convolutional Neural Network Architecture Design