Abstract:Neural Architecture Search (NAS) is a powerful tool for automating effective image and video processing DNN designing. The ranking of the accuracy has been advocated to design an efficient performance predictor for NAS. The previous contrastive method solves the ranking problem by comparing pairs of architectures and predicting their relative performance. However, it only focuses on the rankings between the two involved architectures and neglects the overall quality distributions of the search space, which may suffer generalization issues. On the contrary, we propose to let the performance predictor concentrate on the global quality level of specific architecture, and learn the tier embeddings of the whole search space automatically with learnable queries. The proposed method, dubbed as Neural Architecture Ranker with Query-to-Tier technique (NARQ2T), explores the quality tiers of the search space globally and classifies each individual to the tier they belong to. Thus, the predictor gains knowledge of the performance distributions of the search space which helps to generalize its ranking ability to the datasets more easily. Thanks to the encoder-decoder design, our method is able to predict the latency of the searched model without deteriorating the performance prediction. Meanwhile, the global quality distribution facilitates the search phase by directly sampling candidates according to the statistics of quality tiers, which is free of training a search algorithm, e.g., Reinforcement Learning or Evolutionary Algorithm, thus it simplifies the NAS pipeline and saves the computational overheads. The proposed NARQ2T achieves state-of-the-art performance on two widely used datasets for NAS research. Moreover, extensive experiments have validated the efficacy of the designed method.

A Semi-Supervised Assessor of Neural Architectures

Semi-supervised accuracy predictor-based multi-objective neural architecture search

Semi-Supervised Neural Architecture Search.

Evolutionary Neural Architecture Search with Semi-supervised Accuracy Predictor

Latency-aware Neural Architecture Performance Predictor with Query-to-Tier Technique

Self-supervised Representation Learning for Evolutionary Neural Architecture Search

ReNAS:Relativistic Evaluation of Neural Architecture Search

Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search

Prioritized Architecture Sampling with Monto-Carlo Tree Search

Bridge the Gap Between Architecture Spaces via A Cross-Domain Predictor

Advances in neural architecture search

NeuralMAE: Data-Efficient Neural Architecture Predictor with Masked Autoencoder.

Work-in-Progress: NAPMAE: Generalized Data-Efficient Neural Architecture Predictor with Masked Autoencoder

A Technical View on Neural Architecture Search

Architecture Self-Attention Mechanism: Nonlinear Optimization for Neural Architecture Search

Performance Prediction Based on Neural Architecture Features.

Automating Neural Architecture Design without Search

CAP: A Context-Aware Neural Predictor for NAS

Latent Space Neural Architecture Search via LambdaNDCGloss-Based Listwise Ranker

Accuracy Prediction with Non-neural Model for Neural Architecture Search

Surrogate-assisted Multi-objective Neural Architecture Search for Real-time Semantic Segmentation