Abstract:Over the last decade, deep neural networks have shown great success in the fields of machine learning and computer vision. Currently, the CNN (convolutional neural network) is one of the most successful networks, having been applied in a wide variety of application domains, including pattern recognition, medical diagnosis and signal processing. Despite CNNs’ impressive performance, their architectural design remains a significant challenge for researchers and practitioners. The problem of selecting hyperparameters is extremely important for these networks. The reason for this is that the search space grows exponentially in size as the number of layers increases. In fact, all existing classical and evolutionary pruning methods take as input an already pre-trained or designed architecture. None of them take pruning into account during the design process. However, to evaluate the quality and possible compactness of any generated architecture, filter pruning should be applied before the communication with the data set to compute the classification error. For instance, a medium-quality architecture in terms of classification could become a very light and accurate architecture after pruning, and vice versa. Many cases are possible, and the number of possibilities is huge. This motivated us to frame the whole process as a bi-level optimization problem where: (1) architecture generation is done at the upper level (with minimum NB and NNB) while (2) its filter pruning optimization is done at the lower level. Motivated by evolutionary algorithms’ (EAs) success in bi-level optimization, we use the newly suggested co-evolutionary migration-based algorithm (CEMBA) as a search engine in this research to address our bi-level architectural optimization problem. The performance of our suggested technique, called Bi-CNN-D-C (Bi-level convolution neural network design and compression), is evaluated using the widely used benchmark data sets for image classification, called CIFAR-10, CIFAR-100 and ImageNet. Our proposed approach is validated by means of a set of comparative experiments with respect to relevant state-of-the-art architectures.

Design a multi-objective optimization with dynamic and global filter pruning strategy for convolutional neural network

Pruning by Training: A Novel Deep Neural Network Compression Framework for Image Processing.

Pruning Deep Convolutional Neural Networks Architectures with Evolution Strategy

Batch-Normalization-based Soft Filter Pruning for Deep Convolutional Neural Networks

Multi-Objective Compression for CNNs via Evolutionary Algorithm

Convolutional neural network pruning based on multi-objective feature map selection for image classification

Joint filter and channel pruning of convolutional neural networks as a bi-level optimization problem

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Accelerating Convolutional Networks via Global & Dynamic Filter Pruning

Filter Pruning Via Feature Map Clustering.

Compression of Convolutional Neural Networks With Divergent Representation of Filters

GenExp: Multi-objective pruning for deep neural network based on genetic algorithm

A Knee-Guided Evolutionary Algorithm for Compressing Deep Neural Networks

Convolutional Neural Network Optimization Using Modified NSGA-II

A Greedy Hierarchical Approach to Whole-Network Filter- Pruning in CNNs

A Fast Evolutionary Learning to Optimize CNN

Joint design and compression of convolutional neural networks as a Bi-level optimization problem

Multiobjective Evolutionary Pruning of Deep Neural Networks with Transfer Learning for improving their Performance and Robustness

Pruning Filters while Training for Efficiently Optimizing Deep Learning Networks

Frequency-Domain Dynamic Pruning for Convolutional Neural Networks

Accelerating Convolutional Neural Networks By Group-Wise 2d-Filter Pruning