Abstract:Neural Architecture Search (NAS) is a technique for finding suitable neural network architecture models for given applications. Previously, such search methods are usually based on reinforcement learning, with a recurrent neural network to generate neural network models. However, most NAS methods aim to find a set of candidates with best cost-performance ratios, e.g. high accuracy and low computing time, based on rough estimates derived from the workload generically. As today's deep learning chips accelerate neural network operations with a variety of hardware tricks such as vectors and low-precision data formats, the estimated metrics derived from generic computing operations such as float-point operations (FLOPS) would be very different from the actual latency, throughput, power consumption, etc., which are highly sensitive to the hardware design and even the software optimization in edge AI applications. Thus, instead of taking a long time to pick and train so called good candidates repeatedly based on unreliable estimates, we propose a NAS framework which accelerates the search process by including the actual performance measurements in the search process. The inclusion of actual measurements enables the proposed NAS framework to find candidates based on correct information and reduce the possibility of selecting wrong candidates and wasting search time on wrong candidates. To illustrate the effectiveness of our framework, we prototyped the framework to work with Intel OpenVINO and Field Programmable Gate Arrays (FPGA) to meet the accuracy and latency required by the user. The framework takes the dataset, accuracy and latency requirements from the user and automatically search for candidates to meet the requirements. Case studies and experimental results are presented in this paper to evaluate the effectiveness of our framework for Edge AI applications in real-time image classification.

Co-Exploring Neural Architecture and Network-on-Chip Design for Real-Time Artificial Intelligence

Hardware/Software Co-Exploration of Neural Architectures.

HSCoNAS: Hardware-Software Co-Design of Efficient DNNs Via Neural Architecture Search

Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search

HAO: Hardware-aware neural Architecture Optimization for Efficient Inference

Device-Circuit-Architecture Co-Exploration for Computing-in-Memory Neural Accelerators

Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks

Tiny Neural Network Search and Implementation for Embedded FPGA: a Software-Hardware Co-Design Approach

Hardware-Aware NAS Framework with Layer Adaptive Scheduling on Embedded System

Designing Efficient DNNs Via Hardware-Aware Neural Architecture Search and Beyond

DCNN Search and Accelerator Co-Design: Improve the Adaptability Between NAS Frameworks and Embedded Platforms

Hardware-aware Neural Architecture Search for Stochastic Computing-Based Neural Networks on Tiny Devices

Toward Fast Platform-Aware Neural Architecture Search for FPGA-Accelerated Edge AI Applications

RHNAS: Realizable Hardware and Neural Architecture Search

Algorithm and Hardware Co-design for Reconfigurable CNN Accelerator

Hardware-Aware Neural Architecture Search: Survey and Taxonomy

EdgeNAS: Discovering Efficient Neural Architectures for Edge Systems

Unleashing Network/Accelerator Co-Exploration Potential on FPGAs: A Deeper Joint Search

Hardware-aware One-Shot Neural Architecture Search in Coordinate Ascent Framework

Fast Hardware-Aware Neural Architecture Search

Automated and Holistic Co-design of Neural Networks and ASICs for Enabling In-Pixel Intelligence