Abstract:Modern convolutional networks such as ResNet and NASNet have achieved state-of-the-art results in many computer vision applications. These architectures consist of stages, which are sets of layers that operate on representations in the same resolution. It has been demonstrated that increasing the number of layers in each stage improves the prediction ability of the network. However, the resulting architecture becomes computationally expensive in terms of floating point operations, memory requirements and inference time. Thus, significant human effort is necessary to evaluate different trade-offs between depth and performance. To handle this problem, recent works have proposed to automatically design high-performance architectures, mainly by means of neural architecture search (NAS). Current NAS strategies analyze a large set of possible candidate architectures and, hence, require vast computational resources and take many GPUs days. Motivated by this, we propose a NAS approach to efficiently design accurate and low-cost convolutional architectures and demonstrate that an efficient strategy for designing these architectures is to learn the depth stage-by-stage. For this purpose, our approach increases depth incrementally in each stage taking into account its importance, such that stages with low importance are kept shallow while stages with high importance become deeper. We conduct experiments on the CIFAR and different versions of ImageNet datasets, where we show that architectures discovered by our approach achieve better accuracy and efficiency than human-designed architectures. Additionally, we show that architectures discovered on CIFAR-10 can be successfully transferred to large datasets. Compared to previous NAS approaches, our method is substantially more efficient, as it evaluates one order of magnitude fewer models and yields architectures on par with the state-of-the-art.

Training-free Neural Architecture Search on Hybrid Convolution-attention Networks

Training-free Neural Architectural Search on Transformer Via Evaluating Expressivity and Trainability

Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search

CNASV - A Convolutional Neural Architecture Search-Train Prototype for Computer Vision Task.

AttentiveNAS: Improving Neural Architecture Search Via Attentive Sampling

Prioritized Architecture Sampling with Monto-Carlo Tree Search

BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search

Generative Adversarial Neural Architecture Search

Efficient Evolutionary Neural Architecture Search Based on Hybrid Search Space

MNGNAS: Distilling Adaptive Combination of Multiple Searched Networks for One-Shot Neural Architecture Search

Method for Expanding Search Space With Hybrid Operations in DynamicNAS

Automated Search-Space Generation Neural Architecture Search

SNAS: Stochastic Neural Architecture Search

Evolutionary Neural Architecture Search Combining Multi-Branch ConvNet and Improved Transformer

Contrastive Neural Architecture Search with Neural Architecture Comparators

DNAS:A Decoupled Global Neural Architecture Search Method

Semi-Supervised Neural Architecture Search.

XC-NAS: A New Cellular Encoding Approach for Neural Architecture Search of Multi-path Convolutional Neural Networks

A Semi-Supervised Assessor of Neural Architectures

A Survey on Computationally Efficient Neural Architecture Search

Stage-Wise Neural Architecture Search