Abstract:Over the last decade, deep neural networks have shown great success in the fields of machine learning and computer vision. Currently, the CNN (convolutional neural network) is one of the most successful networks, having been applied in a wide variety of application domains, including pattern recognition, medical diagnosis and signal processing. Despite CNNs’ impressive performance, their architectural design remains a significant challenge for researchers and practitioners. The problem of selecting hyperparameters is extremely important for these networks. The reason for this is that the search space grows exponentially in size as the number of layers increases. In fact, all existing classical and evolutionary pruning methods take as input an already pre-trained or designed architecture. None of them take pruning into account during the design process. However, to evaluate the quality and possible compactness of any generated architecture, filter pruning should be applied before the communication with the data set to compute the classification error. For instance, a medium-quality architecture in terms of classification could become a very light and accurate architecture after pruning, and vice versa. Many cases are possible, and the number of possibilities is huge. This motivated us to frame the whole process as a bi-level optimization problem where: (1) architecture generation is done at the upper level (with minimum NB and NNB) while (2) its filter pruning optimization is done at the lower level. Motivated by evolutionary algorithms’ (EAs) success in bi-level optimization, we use the newly suggested co-evolutionary migration-based algorithm (CEMBA) as a search engine in this research to address our bi-level architectural optimization problem. The performance of our suggested technique, called Bi-CNN-D-C (Bi-level convolution neural network design and compression), is evaluated using the widely used benchmark data sets for image classification, called CIFAR-10, CIFAR-100 and ImageNet. Our proposed approach is validated by means of a set of comparative experiments with respect to relevant state-of-the-art architectures.

Benchmark analysis of popular ImageNet classification deep CNN architectures

Benchmark analysis of popular ImageNet classification deep CNN architectures

Research on Image Classification Method of Features of Combinatorial Convolution

Impact of Fully Connected Layers on Performance of Convolutional Neural Networks for Image Classification

A Survey of the Recent Architectures of Deep Convolutional Neural Networks

Deep Learning Using Isotroping, Laplacing, Eigenvalues Interpolative Binding, and Convolved Determinants with Normed Mapping for Large-Scale Image Retrieval

CNNComparator: Comparative Analytics of Convolutional Neural Networks.

Refining Architectures of Deep Convolutional Neural Networks

Towards Better Analysis of Deep Convolutional Neural Networks

Is it enough to optimize CNN architectures on ImageNet?

A Framework for Designing the Architectures of Deep Convolutional Neural Networks

Optimizing Image Classification: Automated Deep Learning Architecture Crafting with Network and Learning Hyperparameter Tuning

DLBench: a comprehensive experimental evaluation of deep learning frameworks

Deep Neural Network Concepts for Classification using Convolutional Neural Network: A Systematic Review and Evaluation

Joint design and compression of convolutional neural networks as a Bi-level optimization problem

A Survey of Model Compression and Acceleration for Deep Neural Networks.

Return of the Devil in the Details: Delving Deep into Convolutional Nets

Automated CNN Architectural Design: A Simple and Efficient Methodology for Computer Vision Tasks

DEEP LEARNING ARCHITECTURE BASED ON CONVOLUTIONAL NEURAL NETWORK (CNN) IN IMAGE CLASSIFICATION

Deep Convolutional Neural Network (CNN) for Large-Scale Images Classification

Comparable Study Of Convolutional Neural Networks In Classification And Feature Extraction Applications