Abstract:Over the last decade, deep neural networks have shown great success in the fields of machine learning and computer vision. Currently, the CNN (convolutional neural network) is one of the most successful networks, having been applied in a wide variety of application domains, including pattern recognition, medical diagnosis and signal processing. Despite CNNs’ impressive performance, their architectural design remains a significant challenge for researchers and practitioners. The problem of selecting hyperparameters is extremely important for these networks. The reason for this is that the search space grows exponentially in size as the number of layers increases. In fact, all existing classical and evolutionary pruning methods take as input an already pre-trained or designed architecture. None of them take pruning into account during the design process. However, to evaluate the quality and possible compactness of any generated architecture, filter pruning should be applied before the communication with the data set to compute the classification error. For instance, a medium-quality architecture in terms of classification could become a very light and accurate architecture after pruning, and vice versa. Many cases are possible, and the number of possibilities is huge. This motivated us to frame the whole process as a bi-level optimization problem where: (1) architecture generation is done at the upper level (with minimum NB and NNB) while (2) its filter pruning optimization is done at the lower level. Motivated by evolutionary algorithms’ (EAs) success in bi-level optimization, we use the newly suggested co-evolutionary migration-based algorithm (CEMBA) as a search engine in this research to address our bi-level architectural optimization problem. The performance of our suggested technique, called Bi-CNN-D-C (Bi-level convolution neural network design and compression), is evaluated using the widely used benchmark data sets for image classification, called CIFAR-10, CIFAR-100 and ImageNet. Our proposed approach is validated by means of a set of comparative experiments with respect to relevant state-of-the-art architectures.

Automatically Design Convolutional Neural Networks by Optimization With Submodularity and Supermodularity

Neural Network Architecture Optimization through Submodularity and Supermodularity.

A Review of Convolutional Neural Network Architectures and Their Optimizations

Automatic Design of Convolutional Neural Network Architectures under Resource Constraints.

Convolutional Neural Network Optimization Using Modified NSGA-II

Architecture evolution of convolutional neural network using monarch butterfly optimization

Resource Constrained Neural Network Architecture Search: Will a Submodularity Assumption Help?

Refining Architectures of Deep Convolutional Neural Networks

Optimizing Convolutional Neural Network Architectures

Optimizing Convolutional Neural Network Architecture

A Framework for Designing the Architectures of Deep Convolutional Neural Networks

Optimizing Recurrent Neural Networks Architectures under Time Constraints.

Automatically Designing CNN Architectures Using the Genetic Algorithm for Image Classification

Accelerating deep learning model development—towards scalable automated architecture generation for optimal model design

Optimal Design of Convolutional Neural Network Architectures Using Teaching-Learning-Based Optimization for Image Classification

Multiobjective Evolutionary Design of Deep Convolutional Neural Networks for Image Classification

Automated CNN Architectural Design: A Simple and Efficient Methodology for Computer Vision Tasks

Joint design and compression of convolutional neural networks as a Bi-level optimization problem

Automatically designing CNN architectures using genetic algorithm for image classification

Completely Automated CNN Architecture Design Based on Blocks