Abstract:Recently, algorithm-hardware co-exploration for neural networks (NNs) has become the key to obtaining high-quality solutions. However, previous efforts for FPGAs focus on neural architecture search (NAS) while lacking hardware architecture search (HAS), thus limiting the full potential of co-design. Although expanding the scope of HAS offers performance potential, the exponentially increased joint search space presents a formidable challenge. To address this, we propose a deep and efficient framework, which jointly searches for Networks and Accelerators for FPGAs in a balanced co-search space. First, we adjust the NAS space and then introduce a block-level bitwidth search on the software side. Meanwhile, we design a hardware-friendly quantization algorithm to facilitate hardware efficiency and accuracy. Second, we design a dataflow-configurable hardware unit with computation and memory access optimizations for quantized multiplication. Based on this, we incorporate critical heterogeneous multicore architecture exploration on the hardware side. Third, to enable rapid hardware feedback in the enlarged HAS space, we perform resource and performance modeling and design a fast hardware generation algorithm based on the genetic algorithm. Specifically, we apply optimization techniques, like mapping space pruning, greedy bandwidth allocation, and coarse-grained search, to speed up this process. We validate in edge and cloud scenarios. Experimental results show that efficiently explores a significantly larger joint space and provides high-quality solutions. Compared with previous state-of-the-art co-design works, the searched CNN-accelerator pairs improve the throughput by 2.07× ~ 7.10× and energy efficiency by 1.41× ~ 2.27× under similar accuracy on the ImageNet dataset.

From model to FPGA: Software-hardware co-design for efficient neural network acceleration

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

FPGA based cellular neural network optimization: from design space to system.

A Survey of FPGA-Based Neural Network Accelerator

[DL] A Survey of FPGA-based Neural Network Inference Accelerators

A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA

An Overview of FPGA Based Deep Learning Accelerators: Challenges and Opportunities.

Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks

Software-Hardware Codesign for Efficient Neural Network Acceleration

Design Exploration of Multi-FPGAs for Accelerating Deep Learning

A Survey of FPGA Based Deep Learning Accelerators: Challenges and Opportunities

Invited: Algorithm-Software-Hardware Co-Design for Deep Learning Acceleration

WPU: A FPGA-based Scalable, Efficient and Software/Hardware Co-design Deep Neural Network Inference Acceleration Processor

NAF: Deeper Network/Accelerator Co-Exploration for Customizing CNNs on FPGA

Synetgy: Algorithm-hardware Co-design for ConvNet Accelerators on Embedded FPGAs

Designing Deep Learning Hardware Accelerator and Efficiency Evaluation

FPGA/DNN Co-Design: An Efficient Design Methodology for IoT Intelligence on the Edge

Towards Agile DNN Accelerator Design Using Incremental Synthesis on FPGAs

Unleashing Network/Accelerator Co-Exploration Potential on FPGAs: A Deeper Joint Search

Design Flow of Accelerating Hybrid Extremely Low Bit-Width Neural Network in Embedded FPGA

An Efficient Reconfigurable Framework for General Purpose CNN-RNN Models on FPGAs