Abstract:Efficient identification of people and objects, segmentation of regions of interest and extraction of relevant data in images, texts, audios and videos are evolving considerably in these past years, which deep learning methods, combined with recent improvements in computational resources, contributed greatly for this achievement. Although its outstanding potential, development of efficient architectures and modules requires expert knowledge and amount of resource time available. In this paper, we propose an evolutionary-based neural architecture search approach for efficient discovery of convolutional models in a dynamic search space, within only 24 GPU hours. With its efficient search environment and phenotype representation, Gene Expression Programming is adapted for network's cell generation. Despite having limited GPU resource time and broad search space, our proposal achieved similar state-of-the-art to manually-designed convolutional networks and also NAS-generated ones, even beating similar constrained evolutionary-based NAS works. The best cells in different runs achieved stable results, with a mean error of 2.82% in CIFAR-10 dataset (which the best model achieved an error of 2.67%) and 18.83% for CIFAR-100 (best model with 18.16%). For ImageNet in the mobile setting, our best model achieved top-1 and top-5 errors of 29.51% and 10.37%, respectively. Although evolutionary-based NAS works were reported to require a considerable amount of GPU time for architecture search, our approach obtained promising results in little time, encouraging further experiments in evolutionary-based NAS, for search and network representation improvements.

Compute-Efficient Neural Network Architecture Optimization by a Genetic Algorithm

Genetic algorithm for neural network architecture optimization

Optimizing Feed-Forward Neural Networks Using Cascaded Genetic Algorithm

Deep Genetic Network

An Aggressive Genetic Programming Approach for Searching Neural Network Structure Under Computational Constraints

Optimizing Neural Architecture Search using Limited GPU Time in a Dynamic Search Space: A Gene Expression Programming Approach

Neural Network Architecture Search with Differentiable Cartesian Genetic Programming for Regression

Efficient Hyperparameter Optimization in Deep Learning Using a Variable Length Genetic Algorithm

Two Novel Performance Improvements for Evolving CNN Topologies

Self-learning Genetic Algorithm for Neural Network Topology Optimization

Evolving Deep Convolutional Neural Networks for Image Classification

An efficient optimization approach for designing machine learning models based on genetic algorithm

Designing Convolutional Neural Network Architecture Using Genetic Algorithms

An algorithmic framework for the optimization of deep neural networks architectures and hyperparameters

A Gradient-Guided Evolutionary Neural Architecture Search

Using Cartesian Genetic Programming Approach with New Crossover Technique to Design Convolutional Neural Networks

Extending MLP ANN hyper-parameters Optimization by using Genetic Algorithm

Topological optimization of the DenseNet with pretrained-weights inheritance and genetic channel selection

Genetic Algorithm Based Deep Learning Neural Network Structure and Hyperparameter Optimization

A new genetic algorithm based evolutionary neural architecture search for image classification