Abstract:Neural Architecture Search (NAS) is a powerful tool for automating effective image and video processing DNN designing. The ranking of the accuracy has been advocated to design an efficient performance predictor for NAS. The previous contrastive method solves the ranking problem by comparing pairs of architectures and predicting their relative performance. However, it only focuses on the rankings between the two involved architectures and neglects the overall quality distributions of the search space, which may suffer generalization issues. On the contrary, we propose to let the performance predictor concentrate on the global quality level of specific architecture, and learn the tier embeddings of the whole search space automatically with learnable queries. The proposed method, dubbed as Neural Architecture Ranker with Query-to-Tier technique (NARQ2T), explores the quality tiers of the search space globally and classifies each individual to the tier they belong to. Thus, the predictor gains knowledge of the performance distributions of the search space which helps to generalize its ranking ability to the datasets more easily. Thanks to the encoder-decoder design, our method is able to predict the latency of the searched model without deteriorating the performance prediction. Meanwhile, the global quality distribution facilitates the search phase by directly sampling candidates according to the statistics of quality tiers, which is free of training a search algorithm, e.g., Reinforcement Learning or Evolutionary Algorithm, thus it simplifies the NAS pipeline and saves the computational overheads. The proposed NARQ2T achieves state-of-the-art performance on two widely used datasets for NAS research. Moreover, extensive experiments have validated the efficacy of the designed method.

NEAR: A Training-Free Pre-Estimator of Machine Learning Model Performance

How Powerful are Performance Predictors in Neural Architecture Search?

GreenMachine: Automatic Design of Zero-Cost Proxies for Energy-Efficient NAS

Latency-aware Neural Architecture Performance Predictor with Query-to-Tier Technique

Zero-Cost Proxies for Lightweight NAS

Pruning Based Training-Free Neural Architecture Search

EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search

HaeNAS: Hardware-Aware Efficient Neural Architecture Search Via Zero-Cost Proxy.

Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search

ReNAS:Relativistic Evaluation of Neural Architecture Search

Extensible Proxy for Efficient NAS

Neural Architecture Search via Two Constant Shared Weights Initialisations

Surprisingly Strong Performance Prediction with Neural Graph Features

EcoNAS: Finding Proxies for Economical Neural Architecture Search

Zero-Shot Proxy with Incorporated-Score for Lightweight Deep Neural Architecture Search

Neural Architecture Search via Trainless Pruning Algorithm: A Bayesian Evaluation of a Network with Multiple Indicators

AZ-NAS: Assembling Zero-Cost Proxies for Network Architecture Search

Evaluating Efficient Performance Estimators of Neural Architectures

Speedy Performance Estimation for Neural Architecture Search

GENNAPE: Towards Generalized Neural Architecture Performance Estimators

Proxy Data Generation for Fast and Efficient Neural Architecture Search