Abstract:Recently, algorithm-hardware (HW) co-exploration for neural networks (NNs) has become the key to obtaining high-quality solutions. However, previous efforts for field-programmable gate arrays (FPGAs) focus on neural architecture search (NAS) while lacking HW architecture search (HAS), thus limiting the full potential of co-design. Although expanding the scope of HAS offers performance potential, the exponentially increased joint search space presents a formidable challenge. To address this, we propose a deep and efficient framework NAF, which jointly searches for Networks and Accelerators for FPGAs in a balanced co-search space. First, we adjust the NAS space and then introduce a block-level bitwidth search on the software side. Meanwhile, we design a HW-friendly quantization algorithm to facilitate HW efficiency and accuracy. Second, we design a dataflow-configurable HW unit with computation and memory access optimizations for quantized multiplication. Based on this, we incorporate critical heterogeneous multicore architecture exploration on the HW side. Third, to enable rapid HW feedback in the enlarged HAS space, we perform resource and performance modeling and design a fast HW generation algorithm based on the genetic algorithm. Specifically, we apply optimization techniques, like mapping space pruning, greedy bandwidth allocation, and coarse-grained search, to speed up this process. We validate NAF in edge and cloud scenarios. Experimental results show that NAF efficiently explores a significantly larger joint space and provides high-quality solutions. Compared with previous state-of-the-art co-design works, the searched convolutional neural network-accelerator pairs improve the throughput by $2.07\times \sim 7.10\times $ and energy efficiency by $1.41\times \sim 2.27\times $ under similar accuracy on the ImageNet dataset.

HAO: Hardware-aware neural Architecture Optimization for Efficient Inference

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search

Co-Exploring Neural Architecture and Network-on-Chip Design for Real-Time Artificial Intelligence

NAF: Deeper Network/Accelerator Co-Exploration for Customizing CNNs on FPGA

HSCoNAS: Hardware-Software Co-Design of Efficient DNNs Via Neural Architecture Search

Unleashing Network/Accelerator Co-Exploration Potential on FPGAs: A Deeper Joint Search

Designing Efficient DNNs Via Hardware-Aware Neural Architecture Search and Beyond

[DL] A Survey of FPGA-based Neural Network Inference Accelerators

Invited: Algorithm-Software-Hardware Co-Design for Deep Learning Acceleration

Hardware/Software Co-Exploration of Neural Architectures.

An FPGA-Based Resource-Saving Hardware Accelerator for Deep Neural Network

HaeNAS: Hardware-Aware Efficient Neural Architecture Search Via Zero-Cost Proxy.

A General Neural Network Hardware Architecture on FPGA

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

NASH: Neural Architecture Search for Hardware-Optimized Machine Learning Models

Tiny Neural Network Search and Implementation for Embedded FPGA: a Software-Hardware Co-Design Approach

A neural network accelerated optimization method for FPGA

Design Implementation of FPGA-Based Neural Network Acceleration

FPGA/DNN Co-Design: An Efficient Design Methodology for IoT Intelligence on the Edge