Abstract:With the rise of various smart electronics and mobile/edge devices, many existing high-accuracy convolutional neural network (CNN) models are difficult to be applied in practice due to the limited resources, such as memory capacity, power consumption, and spectral efficiency. In order to meet these constraints, researchers have carefully designed some lightweight networks. Meanwhile, to reduce the reliance on manual design on expert experience, some researchers also work to improve neural architecture search (NAS) algorithms to automatically design small networks, exploiting the multiobjective approaches that consider both accuracy and other important goals during optimization. However, simply searching for smaller network models is not consistent with the current research belief of “the deeper the better” and may affect the effectiveness of the model and thus waste the limited resources available. In this article, we propose an automatic method for designing CNNs architectures under constraint handling, which can search for optimal network models meeting the preset constraint. Specifically, an adaptive penalty algorithm is used for fitness evaluation, and a selective repair operation is developed for infeasible individuals to search for feasible CNN architectures. As a case study, we set the complexity (the number of parameters) as a resource constraint and perform multiple experiments on CIFAR-10 and CIFAR-100, to demonstrate the effectiveness of the proposed method. In addition, the proposed algorithm is compared with a state-of-the-art algorithm, NSGA-Net, and several manual-designed models. The experimental results show that the proposed algorithm can successfully solve the problem of the uncertain size of the optimal CNN model under the random search strategy, and the automatically designed CNN model can satisfy the predefined resource constraint while achieving better accuracy.

Optimizing Convolutional Neural Network Architecture

Optimizing Convolutional Neural Network Architectures

Structured Pruning for Efficient Convolutional Neural Networks Via Incremental Regularization

A Pruning Method Based on the Dissimilarity of Angle among Channels and Filters

Sensitivity-Oriented Layer-Wise Acceleration and Compression for Convolutional Neural Network.

Refining Architectures of Deep Convolutional Neural Networks

Loss-Driven Channel Pruning of Convolutional Neural Networks.

Learning Low Resource Consumption CNN through Pruning and Quantization

Learning Efficient Convolutional Networks Through Network Slimming.

Prune the Convolutional Neural Networks with Sparse Shrink

A Review of Convolutional Neural Network Architectures and Their Optimizations

Automatic Design of Convolutional Neural Network Architectures under Resource Constraints.

Complexity-Driven CNN Compression for Resource-constrained Edge AI

Optimizing Dense Feed-Forward Neural Networks

Structure Level Pruning of Efficient Convolutional Neural Networks with Sparse Group LASSO

Automatic channel pruning via clustering and swarm intelligence optimization for CNN

A Framework for Designing the Architectures of Deep Convolutional Neural Networks

On pruned, quantized and compact CNN architectures for vision applications: an empirical study

Accelerating Convolutional Neural Networks by Removing Interspatial and Interkernel Redundancies.

SlimConv: Reducing Channel Redundancy in Convolutional Neural Networks by Features Recombining

CCPrune: Collaborative Channel Pruning for Learning Compact Convolutional Networks