Abstract:With the rise of various smart electronics and mobile/edge devices, many existing high-accuracy convolutional neural network (CNN) models are difficult to be applied in practice due to the limited resources, such as memory capacity, power consumption, and spectral efficiency. In order to meet these constraints, researchers have carefully designed some lightweight networks. Meanwhile, to reduce the reliance on manual design on expert experience, some researchers also work to improve neural architecture search (NAS) algorithms to automatically design small networks, exploiting the multiobjective approaches that consider both accuracy and other important goals during optimization. However, simply searching for smaller network models is not consistent with the current research belief of “the deeper the better” and may affect the effectiveness of the model and thus waste the limited resources available. In this article, we propose an automatic method for designing CNNs architectures under constraint handling, which can search for optimal network models meeting the preset constraint. Specifically, an adaptive penalty algorithm is used for fitness evaluation, and a selective repair operation is developed for infeasible individuals to search for feasible CNN architectures. As a case study, we set the complexity (the number of parameters) as a resource constraint and perform multiple experiments on CIFAR-10 and CIFAR-100, to demonstrate the effectiveness of the proposed method. In addition, the proposed algorithm is compared with a state-of-the-art algorithm, NSGA-Net, and several manual-designed models. The experimental results show that the proposed algorithm can successfully solve the problem of the uncertain size of the optimal CNN model under the random search strategy, and the automatically designed CNN model can satisfy the predefined resource constraint while achieving better accuracy.

Should You Go Deeper? Optimizing Convolutional Neural Network Architectures without Training by Receptive Field Analysis

Receptive Field Refinement for Convolutional Neural Networks Reliably Improves Predictive Performance

Optimizing Convolutional Neural Network Architecture

A Framework for Designing the Architectures of Deep Convolutional Neural Networks

Refining Architectures of Deep Convolutional Neural Networks

Optimizing Convolutional Neural Network Architectures

Sensitivity-Oriented Layer-Wise Acceleration and Compression for Convolutional Neural Network.

Towards Better Analysis of Deep Convolutional Neural Networks

Layer-Specific Optimization: Sensitivity Based Convolution Layers Basis Search

Layer-Wise Training To Create Efficient Convolutional Neural Networks

Exploring Deep Reuse in Winograd CNN Inference.

Building Efficient CNNs Using Depthwise Convolutional Eigen-Filters (DeCEF)

Exploring the Design Space of Deep Convolutional Neural Networks at Large Scale

Accelerating Convolutional Neural Networks by Removing Interspatial and Interkernel Redundancies.

Deeper Interpretability of Deep Networks

Convolutional Neural Networks Exploiting Attributes of Biological Neurons

The Shallow End: Empowering Shallower Deep-Convolutional Networks through Auxiliary Outputs

Optimizing Memory Efficiency for Deep Convolutional Neural Networks on GPUs.

Toward Efficient Convolutional Neural Networks With Structured Ternary Patterns

Computational issues in Optimization for Deep networks

Automatic Design of Convolutional Neural Network Architectures under Resource Constraints.