Abstract:Neural Architecture Search (NAS) is a powerful tool for automating effective image and video processing DNN designing. The ranking of the accuracy has been advocated to design an efficient performance predictor for NAS. The previous contrastive method solves the ranking problem by comparing pairs of architectures and predicting their relative performance. However, it only focuses on the rankings between the two involved architectures and neglects the overall quality distributions of the search space, which may suffer generalization issues. On the contrary, we propose to let the performance predictor concentrate on the global quality level of specific architecture, and learn the tier embeddings of the whole search space automatically with learnable queries. The proposed method, dubbed as Neural Architecture Ranker with Query-to-Tier technique (NARQ2T), explores the quality tiers of the search space globally and classifies each individual to the tier they belong to. Thus, the predictor gains knowledge of the performance distributions of the search space which helps to generalize its ranking ability to the datasets more easily. Thanks to the encoder-decoder design, our method is able to predict the latency of the searched model without deteriorating the performance prediction. Meanwhile, the global quality distribution facilitates the search phase by directly sampling candidates according to the statistics of quality tiers, which is free of training a search algorithm, e.g., Reinforcement Learning or Evolutionary Algorithm, thus it simplifies the NAS pipeline and saves the computational overheads. The proposed NARQ2T achieves state-of-the-art performance on two widely used datasets for NAS research. Moreover, extensive experiments have validated the efficacy of the designed method.

Ranking‐based architecture generation for surrogate‐assisted neural architecture search

Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search

A neural architecture generator for efficient search space

Similarity surrogate-assisted evolutionary neural architecture search with dual encoding strategy

Latency-aware Neural Architecture Performance Predictor with Query-to-Tier Technique

Generative Adversarial Neural Architecture Search

Learning to Rank Ace Neural Architectures via Normalized Discounted Cumulative Gain

Prioritized Architecture Sampling with Monto-Carlo Tree Search

Pareto-wise Ranking Classifier for Multi-objective Evolutionary Neural Architecture Search

GPT-NAS: Evolutionary Neural Architecture Search with the Generative Pre-Trained Model

Semi-supervised accuracy predictor-based multi-objective neural architecture search

Posterior-Guided Neural Architecture Search

Self-supervised Representation Learning for Evolutionary Neural Architecture Search

Boosting Order-Preserving and Transferability for Neural Architecture Search: a Joint Architecture Refined Search and Fine-tuning Approach

Guided evolutionary Neural Architecture Search with efficient performance estimation

Action Command Encoding for Surrogate-Assisted Neural Architecture Search

Neural Architecture Search with Representation Mutual Information

Evaluating the Search Phase of Neural Architecture Search

Rethinking Bi-Level Optimization in Neural Architecture Search: A Gibbs Sampling Perspective.

Understanding and Improving One-shot Neural Architecture Optimization

A Semi-Supervised Assessor of Neural Architectures