Abstract:Recently, algorithm-hardware co-exploration for neural networks (NNs) has become the key to obtaining high-quality solutions. However, previous efforts for FPGAs focus on neural architecture search (NAS) while lacking hardware architecture search (HAS), thus limiting the full potential of co-design. Although expanding the scope of HAS offers performance potential, the exponentially increased joint search space presents a formidable challenge. To address this, we propose a deep and efficient framework, which jointly searches for Networks and Accelerators for FPGAs in a balanced co-search space. First, we adjust the NAS space and then introduce a block-level bitwidth search on the software side. Meanwhile, we design a hardware-friendly quantization algorithm to facilitate hardware efficiency and accuracy. Second, we design a dataflow-configurable hardware unit with computation and memory access optimizations for quantized multiplication. Based on this, we incorporate critical heterogeneous multicore architecture exploration on the hardware side. Third, to enable rapid hardware feedback in the enlarged HAS space, we perform resource and performance modeling and design a fast hardware generation algorithm based on the genetic algorithm. Specifically, we apply optimization techniques, like mapping space pruning, greedy bandwidth allocation, and coarse-grained search, to speed up this process. We validate in edge and cloud scenarios. Experimental results show that efficiently explores a significantly larger joint space and provides high-quality solutions. Compared with previous state-of-the-art co-design works, the searched CNN-accelerator pairs improve the throughput by 2.07× ~ 7.10× and energy efficiency by 1.41× ~ 2.27× under similar accuracy on the ImageNet dataset.

Hardware/Software Co-Exploration of Neural Architectures.

HSCoNAS: Hardware-Software Co-Design of Efficient DNNs Via Neural Architecture Search

Co-Exploring Neural Architecture and Network-on-Chip Design for Real-Time Artificial Intelligence

Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search

Device-Circuit-Architecture Co-Exploration for Computing-in-Memory Neural Accelerators

Unleashing Network/Accelerator Co-Exploration Potential on FPGAs: A Deeper Joint Search

Hardware-Aware NAS Framework with Layer Adaptive Scheduling on Embedded System

Hardware-aware One-Shot Neural Architecture Search in Coordinate Ascent Framework

HAO: Hardware-aware neural Architecture Optimization for Efficient Inference

Hardware Aware Evolutionary Neural Architecture Search using Representation Similarity Metric

Fast Hardware-Aware Neural Architecture Search

Beyond Training: A Zero-Shot Framework to Neural Architecture and Accelerator Co-Exploration

Designing Efficient DNNs Via Hardware-Aware Neural Architecture Search and Beyond

Multi-Objective Hardware Aware Neural Architecture Search using Hardware Cost Diversity

DCNN Search and Accelerator Co-Design: Improve the Adaptability Between NAS Frameworks and Embedded Platforms

HaeNAS: Hardware-Aware Efficient Neural Architecture Search Via Zero-Cost Proxy.

SurgeNAS: A Comprehensive Surgery on Hardware-Aware Differentiable Neural Architecture Search

Integrating Hardware Diversity with Neural Architecture Search for Efficient Convolutional Neural Networks

RHNAS: Realizable Hardware and Neural Architecture Search

Advances in neural architecture search