Abstract:Neural Architecture Search (NAS) is a powerful tool for automating effective image and video processing DNN designing. The ranking of the accuracy has been advocated to design an efficient performance predictor for NAS. The previous contrastive method solves the ranking problem by comparing pairs of architectures and predicting their relative performance. However, it only focuses on the rankings between the two involved architectures and neglects the overall quality distributions of the search space, which may suffer generalization issues. On the contrary, we propose to let the performance predictor concentrate on the global quality level of specific architecture, and learn the tier embeddings of the whole search space automatically with learnable queries. The proposed method, dubbed as Neural Architecture Ranker with Query-to-Tier technique (NARQ2T), explores the quality tiers of the search space globally and classifies each individual to the tier they belong to. Thus, the predictor gains knowledge of the performance distributions of the search space which helps to generalize its ranking ability to the datasets more easily. Thanks to the encoder-decoder design, our method is able to predict the latency of the searched model without deteriorating the performance prediction. Meanwhile, the global quality distribution facilitates the search phase by directly sampling candidates according to the statistics of quality tiers, which is free of training a search algorithm, e.g., Reinforcement Learning or Evolutionary Algorithm, thus it simplifies the NAS pipeline and saves the computational overheads. The proposed NARQ2T achieves state-of-the-art performance on two widely used datasets for NAS research. Moreover, extensive experiments have validated the efficacy of the designed method.

NeuralMAE: Data-Efficient Neural Architecture Predictor with Masked Autoencoder.

Work-in-Progress: NAPMAE: Generalized Data-Efficient Neural Architecture Predictor with Masked Autoencoder

Masked Autoencoders for Point Cloud Self-supervised Learning.

Masked Autoencoders Are Robust Neural Architecture Search Learners

Latency-aware Neural Architecture Performance Predictor with Query-to-Tier Technique

A Semi-Supervised Assessor of Neural Architectures

CAP: A Context-Aware Neural Predictor for NAS

Self-supervised Representation Learning for Evolutionary Neural Architecture Search

Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search

Efficient Sampling for Predictor-Based Neural Architecture Search

Accuracy Prediction with Non-neural Model for Neural Architecture Search

Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS "Cold-Start"

Single-Domain Generalized Predictor for Neural Architecture Search System

Homogeneous Architecture Augmentation for Neural Predictor

How Powerful are Performance Predictors in Neural Architecture Search?

Semi-supervised accuracy predictor-based multi-objective neural architecture search

Data Aware Neural Architecture Search

ReNAS:Relativistic Evaluation of Neural Architecture Search

Advances in neural architecture search

Neural Architecture Search with Representation Mutual Information

FR-NAS: Forward-and-Reverse Graph Predictor for Efficient Neural Architecture Search