Abstract:Convolutional neural networks (CNNs) are the backbones of deep learning paradigms for numerous vision tasks. Early advancements in CNN architectures are primarily driven by human expertise and by elaborate design processes. Recently, neural architecture search was proposed with the aim of automating the network design process and generating task-dependent architectures. While existing approaches have achieved competitive performance in image classification, they are not well suited to problems where the computational budget is limited for two reasons: 1) the obtained architectures are either solely optimized for classification performance, or only for one deployment scenario and 2) the search process requires vast computational resources in most approaches. To overcome these limitations, we propose an evolutionary algorithm for searching neural architectures under multiple objectives, such as classification performance and floating point operations (FLOPs). The proposed method addresses the first shortcoming by populating a set of architectures to approximate the entire Pareto frontier through genetic operations that recombine and modify architectural components progressively. Our approach improves computational efficiency by carefully down-scaling the architectures during the search as well as reinforcing the patterns commonly shared among past successful architectures through Bayesian model learning. The integration of these two main contributions allows an efficient design of architectures that are competitive and in most cases outperform both manually and automatically designed architectures on benchmark image classification datasets: CIFAR, ImageNet, and human chest X-ray. The flexibility provided from simultaneously obtaining multiple architecture choices for different compute requirements further differentiates our approach from other methods in the literature.

Particle swarm optimization of deep neural networks architectures for image classification

Evolving Deep Convolutional Neural Networks by Variable-Length Particle Swarm Optimization for Image Classification

Evolving Deep Architecture Generation with Residual Connections for Image Classification Using Particle Swarm Optimization

Particle Swarm Optimisation for Evolving Deep Neural Networks for Image Classification by Evolving and Stacking Transferable Blocks

Efficient Hyperparameter Optimization for Convolution Neural Networks in Deep Learning: A Distributed Particle Swarm Optimization Approach

Neural Architecture Search using Particle Swarm and Ant Colony Optimization

A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks

PSO-Convolutional Neural Networks with Heterogeneous Learning Rate

Particle swarm optimization-based automatic parameter selection for deep neural networks and its applications in large-scale and high-dimensional data

Designing neural networks using hybrid particle swarm optimization

Optimization of Convolutional Neural Network Using the Linearly Decreasing Weight Particle Swarm Optimization

Hyper-parameter selection in deep neural networks using parallel particle swarm optimization

Parameters Optimization of Deep Learning Models using Particle Swarm Optimization

Surrogate-assisted Particle Swarm Optimisation for Evolving Variable-length Transferable Blocks for Image Classification

Multiobjective Evolutionary Design of Deep Convolutional Neural Networks for Image Classification

SEM-Net: Deep features selections with Binary Particle Swarm Optimization Method for classification of scanning electron microscope images

A Particle Swarm Optimization-based Flexible Convolutional Auto-Encoder for Image Classification

Video deepfake detection using Particle Swarm Optimization improved deep neural networks

Convolutional neural network with group theory and random selection particle swarm optimizer for enhancing cancer image classification