Abstract:Deep neural networks, specifically deep convolutional neural networks (DCNNs), have been highly successful in machine learning and computer vision, but a significant challenge when using these networks is choosing the right hyperparameters. As the number of layers in the network increases, the search space also becomes larger. To overcome this issue, researchers in deep learning have suggested using deep compression techniques to decrease memory usage and computational complexity. In this paper, we present a new approach for compressing deep CNNs by combining filter and channel pruning methods based on Evolutionary Algorithms (EA). This method involves eliminating filters and channels in order to decrease the number of parameters and computational complexity of the model. Additionally, we propose a bi-level optimization problem that interacts between the hyperparameters of the convolution layer. Bi-level optimization problems are known to be difficult as they involve two levels of optimization tasks, where only the optimal solutions to the lower-level problem are considered as feasible candidates for the upper-level problem. In this work, the upper-level problem is represented by a set of filters to be pruned in order to minimize the number of selected filters, while the lower-level problem is represented by a set of channels to be pruned in order to minimize the number of selected channels per filter. Our research has focused on developing a new method for solving bi-level problems, which we have named Bi-CNN-Pruning. To achieve this, we have adopted the Co-Evolutionary Migration-Based Algorithm (CEMBA) as our search engine. The Bi-CNN-Pruning method is then evaluated using image classification benchmarks on well-known datasets such as CIFAR-10 and CIFAR-100. The results of our evaluation demonstrate that our bi-level proposal outperforms state-of-the-art architectures, and we provide a detailed analysis of the results using commonly employed performance metrics.

Multi-Objective Compression for CNNs via Evolutionary Algorithm

Compression of deep neural networks: bridging the gap between conventional-based pruning and evolutionary approach

Pruning by Training: A Novel Deep Neural Network Compression Framework for Image Processing.

Multi-Objective Net Architecture Pruning for Remote Sensing Classification.

Neural Network Pruning by Cooperative Coevolution

Evolutionary Multi-Objective One-Shot Filter Pruning for Designing Lightweight Convolutional Neural Network.

Towards Evolutionary Compression.

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

A Knee-Guided Evolutionary Algorithm for Compressing Deep Neural Networks

Pruning Deep Convolutional Neural Networks Architectures with Evolution Strategy

Joint filter and channel pruning of convolutional neural networks as a bi-level optimization problem

Differential Evolution Based Layer-Wise Weight Pruning for Compressing Deep Neural Networks

An Entropy-based Pruning Method for CNN Compression

Design a multi-objective optimization with dynamic and global filter pruning strategy for convolutional neural network

Convolutional neural network pruning based on multi-objective feature map selection for image classification

Multi-grained Pruning Method of Convolutional Neural Network.

Multiobjective Evolutionary Pruning of Deep Neural Networks with Transfer Learning for improving their Performance and Robustness

AACP: Model Compression by Accurate and Automatic Channel Pruning.

Filter Pruning Via Separation of Sparsity Search and Model Training

Network Pruning Using Sparse Learning and Genetic Algorithm

Evolutionary channel pruning for real-time object detection