Across-task Neural Architecture Search Via Meta Learning

Jingtao Rong,Xinyi Yu,Mingyang Zhang,Linlin Ou
DOI: https://doi.org/10.1007/s13042-022-01678-5
2023-01-01
International Journal of Machine Learning and Cybernetics
Abstract:Adequate labeled data and expensive compute resources are the prerequisites for the success of neural architecture search (NAS). It is challenging to apply NAS in meta-learning scenarios with limited compute resources and data. In this paper, an across-task neural architecture search (AT-NAS) method is proposed to address this problem via combining gradient-based meta-learning with EA-based NAS to learn over the distribution of tasks. The supernet is learned over an entire set of tasks by meta learning. Architecture encodes of subnets sampled from the supernet are iteratively adapted by evolutionary algorithms while simultaneously searching for a task-sensitive meta-network. The searched meta-network can adapt to a novel task via a few learning steps and it only costs a little search time. Empirical results show that AT-NAS achieves excellent performance in few-shot classification. The performance of AT-NAS on classification benchmarks is comparable to that of models searched from scratch, by adapting the architecture in less than an hour from a 5-GPU-day pretrained meta-network.
What problem does this paper attempt to address?