Abstract:Despite the success of recent Neural Architecture Search (NAS) methods on various tasks which have shown to output networks that largely outperform human-designed networks, conventional NAS methods have mostly tackled the optimization of searching for the network architecture for a single task (dataset), which does not generalize well across multiple tasks (datasets). Moreover, since such task-specific methods search for a neural architecture from scratch for every given task, they incur a large computational cost, which is problematic when the time and monetary budget are limited. In this paper, we propose an efficient NAS framework that is trained once on a database consisting of datasets and pretrained networks and can rapidly search for a neural architecture for a novel dataset. The proposed MetaD2A (Meta Dataset-to-Architecture) model can stochastically generate graphs (architectures) from a given set (dataset) via a cross-modal latent space learned with amortized meta-learning. Moreover, we also propose a meta-performance predictor to estimate and select the best architecture without direct training on target datasets. The experimental results demonstrate that our model meta-learned on subsets of ImageNet-1K and architectures from NAS-Bench 201 search space successfully generalizes to multiple unseen datasets including CIFAR-10 and CIFAR-100, with an average search time of 33 GPU seconds. Even under MobileNetV3 search space, MetaD2A is 5.5K times faster than NSGANetV2, a transferable NAS method, with comparable performance. We believe that the MetaD2A proposes a new research direction for rapid NAS as well as ways to utilize the knowledge from rich databases of datasets and architectures accumulated over the past years. Code is available at <a class="link-external link-https" href="https://github.com/HayeonLee/MetaD2A" rel="external noopener nofollow">this https URL</a>.

Efficient Automatic Meta Optimization Search for Few-Shot Learning

Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning

Meta Navigator: Search for a Good Adaptation Policy for Few-shot Learning

Neural Fine-Tuning Search for Few-Shot Learning

Meta-learning methodology based on meta-unsupervised algorithm for meta-model selection to solve few-shot base-tasks

Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networks

Meta-SGD: Learning to Learn Quickly for Few-Shot Learning

MetaNODE: Prototype Optimization as a Neural ODE for Few-Shot Learning

Learning to Learn by Jointly Optimizing Neural Architecture and Weights

Meta-Learning Adversarial Domain Adaptation Network for Few-Shot Text Classification.

Meta-learning autoencoders for few-shot prediction

Connection-Adaptive Meta-Learning

Multi-Objective Neural Architecture Search by Learning Search Space Partitions

Ranking architectures using meta-learning

Advances in neural architecture search

Unsupervised meta-learning for few-shot learning

Rapid Neural Architecture Search by Learning to Generate Graphs from Datasets

A metric-based meta-learning approach combined attention mechanism and ensemble learning for few-shot learning

FewShotNeRF: Meta-Learning-based Novel View Synthesis for Rapid Scene-Specific Adaptation

Efficient Novelty-Driven Neural Architecture Search