Abstract:Neural predictors have shown great potential in the evaluation process of neural architecture search (NAS). However, current predictor-based approaches overlook the fact that training a predictor necessitates a considerable number of trained neural networks as the labeled training set, which is costly to obtain. Therefore, the critical issue in utilizing predictors for NAS is to train a high-performance predictor using as few trained neural networks as possible. Although some methods attempt to address this problem through unsupervised learning, they often result in inaccurate predictions. We argue that the unsupervised tasks intended for the common graph data are too challenging for neural networks, causing unsupervised training to be susceptible to performance crashes in NAS. To address this issue, we propose a Curricumum-guided Contrastive Learning framework for neural Predictor (DCLP). Our method simplifies the contrastive task by designing a novel curriculum to enhance the stability of unlabeled training data distribution during contrastive training. Specifically, we propose a scheduler that ranks the training data according to the contrastive difficulty of each data and then inputs them to the contrastive learner in order. This approach concentrates the training data distribution and makes contrastive training more efficient. By using our method, the contrastive learner incrementally learns feature representations via unsupervised data on a smooth learning curve, avoiding performance crashes that may occur with excessively variable training data distributions. We experimentally demonstrate that DCLP has high accuracy and efficiency compared with existing predictors, and shows promising potential to discover superior architectures in various search spaces when combined with search strategies. Our code is available at: <a class="link-external link-https" href="https://github.com/Zhengsh123/DCLP" rel="external noopener nofollow">this https URL</a>.

Single-Domain Generalized Predictor for Neural Architecture Search System

Generalized Global Ranking-Aware Neural Architecture Ranker for Efficient Image Classifier Search

Latency-aware Neural Architecture Performance Predictor with Query-to-Tier Technique

Accuracy Prediction with Non-neural Model for Neural Architecture Search

Work-in-Progress: NAPMAE: Generalized Data-Efficient Neural Architecture Predictor with Masked Autoencoder

How Powerful are Performance Predictors in Neural Architecture Search?

Neural Architecture Search with GBDT

Self-supervised Representation Learning for Evolutionary Neural Architecture Search

NeuralMAE: Data-Efficient Neural Architecture Predictor with Masked Autoencoder.

Semi-supervised accuracy predictor-based multi-objective neural architecture search

Efficient Sampling for Predictor-Based Neural Architecture Search

CAP: A Context-Aware Neural Predictor for NAS

FR-NAS: Forward-and-Reverse Graph Predictor for Efficient Neural Architecture Search

TAPAS: Train-less Accuracy Predictor for Architecture Search

DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning

Multi-Predict: Few Shot Predictors For Efficient Neural Architecture Search

A Semi-Supervised Assessor of Neural Architectures

Homogeneous Architecture Augmentation for Neural Predictor

Dynamic Ensemble of Low-fidelity Experts: Mitigating NAS "Cold-Start"

GENNAPE: Towards Generalized Neural Architecture Performance Estimators

$\alpha$ DARTS Once More: Enhancing Differentiable Architecture Search by Masked Image Modeling