Abstract:Neural Architecture Search (NAS) is a powerful tool for automating effective image and video processing DNN designing. The ranking of the accuracy has been advocated to design an efficient performance predictor for NAS. The previous contrastive method solves the ranking problem by comparing pairs of architectures and predicting their relative performance. However, it only focuses on the rankings between the two involved architectures and neglects the overall quality distributions of the search space, which may suffer generalization issues. On the contrary, we propose to let the performance predictor concentrate on the global quality level of specific architecture, and learn the tier embeddings of the whole search space automatically with learnable queries. The proposed method, dubbed as Neural Architecture Ranker with Query-to-Tier technique (NARQ2T), explores the quality tiers of the search space globally and classifies each individual to the tier they belong to. Thus, the predictor gains knowledge of the performance distributions of the search space which helps to generalize its ranking ability to the datasets more easily. Thanks to the encoder-decoder design, our method is able to predict the latency of the searched model without deteriorating the performance prediction. Meanwhile, the global quality distribution facilitates the search phase by directly sampling candidates according to the statistics of quality tiers, which is free of training a search algorithm, e.g., Reinforcement Learning or Evolutionary Algorithm, thus it simplifies the NAS pipeline and saves the computational overheads. The proposed NARQ2T achieves state-of-the-art performance on two widely used datasets for NAS research. Moreover, extensive experiments have validated the efficacy of the designed method.

Work-in-Progress: NAPMAE: Generalized Data-Efficient Neural Architecture Predictor with Masked Autoencoder

NeuralMAE: Data-Efficient Neural Architecture Predictor with Masked Autoencoder.

Masked Autoencoders Are Robust Neural Architecture Search Learners

A Semi-Supervised Assessor of Neural Architectures

Self-supervised Representation Learning for Evolutionary Neural Architecture Search

Latency-aware Neural Architecture Performance Predictor with Query-to-Tier Technique

Single-Domain Generalized Predictor for Neural Architecture Search System

Bridge the Gap Between Architecture Spaces via A Cross-Domain Predictor

CAP: A Context-Aware Neural Predictor for NAS

Evolutionary Neural Architecture Search with Semi-supervised Accuracy Predictor

Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS "Cold-Start"

Accuracy Prediction with Non-neural Model for Neural Architecture Search

Efficient Sampling for Predictor-Based Neural Architecture Search

Homogeneous Architecture Augmentation for Neural Predictor

Semi-supervised accuracy predictor-based multi-objective neural architecture search

Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search

Performance Prediction Based on Neural Architecture Features.

NPENAS: Neural Predictor Guided Evolution for Neural Architecture Search

How Powerful are Performance Predictors in Neural Architecture Search?

Latent Space Neural Architecture Search via LambdaNDCGloss-Based Listwise Ranker