MicroNAS: Zero-Shot Neural Architecture Search for MCUs

Ye Qiao,Haocheng Xu,Yifan Zhang,Sitao Huang

2024-01-17

Abstract:Neural Architecture Search (NAS) effectively discovers new Convolutional Neural Network (CNN) architectures, particularly for accuracy optimization. However, prior approaches often require resource-intensive training on super networks or extensive architecture evaluations, limiting practical applications. To address these challenges, we propose MicroNAS, a hardware-aware zero-shot NAS framework designed for microcontroller units (MCUs) in edge computing. MicroNAS considers target hardware optimality during the search, utilizing specialized performance indicators to identify optimal neural architectures without high computational costs. Compared to previous works, MicroNAS achieves up to 1104x improvement in search efficiency and discovers models with over 3.23x faster MCU inference while maintaining similar accuracy

Machine Learning,Artificial Intelligence

What problem does this paper attempt to address?

The paper aims to address the challenges faced by Neural Architecture Search (NAS) when deployed in edge computing environments, particularly for resource-constrained Microcontroller Units (MCUs). Specifically, traditional NAS methods often require substantial computational resources for supernet training or architecture evaluation, limiting their practicality. To tackle these issues, the authors propose a new framework called MicroNAS.

MicroNAS is a hardware-aware zero-cost NAS framework specifically designed for MCU environments. It identifies the optimal neural network architecture using specialized performance metrics without incurring significant computational costs. Compared to previous NAS methods, MicroNAS improves search efficiency by up to 1104 times and enhances inference speed on MCUs by over 3.23 times while maintaining similar accuracy.

The main contributions of the paper include:
1. **MicroNAS Framework**: Introducing a new zero-cost NAS framework for efficient neural network architecture search on MCUs.
2. **Hybrid Objective Function**: Proposing an objective function that combines neural tangent kernel spectrum, linear region count, and hardware proxies, significantly improving the quality of NAS in MCU environments.
3. **Hardware-Aware Pruning Search Algorithm**: Presenting an innovative pruning search algorithm to enhance search efficiency under resource-constrained conditions.
4. **Experimental Results Analysis**: Demonstrating significant improvements in search efficiency through tests with a training-free proxy on the NAS-Bench-201 space, discovering models with faster inference speeds.

These contributions make MicroNAS an important step towards efficient NAS solutions in edge computing environments.