Abstract:Accurate and efficient Machine Learning algorithms are of vital importance to many problems, especially on classification or clustering tasks but need a universal AI model standard. Unifying machine learning models into a common ecosystem can lead to less development time and better framework interoperability. ONNX (Open Neural Network Exchange Format) is a popular open format to represent deep learning models so that AI developers can more easily move models between state-of-the-art tools. On top of that, hardware companies such as Nvidia or Intel try to keep up with this trend and produce hardware-optimized runtimes (i.e. for CPUs, GPUs, FPGAs) that can handle these open format AI models like ONNX. That enables developers to leverage an heterogeneous mix of hardware and use whichever AI framework they prefer. However, FPGAs have a more challenging solution strategy which as a platform it is also proven to address these kind of problems very efficiently in terms of performance and power. This work is based on an early development stage project which is called HLS4ML originally created for particle physics applications via the automatic generation of neural networks (NNs) for embedded Xilinx FPGAs. Our work involves a hardware-aware NN training and a generalized optimization scheme on top of HLS4ML that boosts the performance and power efficiency of this package and adds functionality for cloud FPGA firmware from any NN model. We start from the FPGA-oriented training of a model in Keras for image recognition, converting into the ONNX open format then porting and optimizing it for cloud FPGAs using a novel scheme with optimizations in host, memory and kernels while using multiple levels of network precision. To the best of our knowledge this is a novel approach that also achieves a speed-up of up to 102<math>×</math> over single CPU in performance and up to 5.5<math>×</math> over GPU in performance/watt.

A Generic Approach for Neural Networks on FPGA

A Near Memory Computing FPGA Architecture for Neural Network Acceleration

FPGA based cellular neural network optimization: from design space to system.

Complex-Exponential-Based Bio-Inspired Neuron Model Implementation in FPGA Using Xilinx System Generator and Vivado Design Suite

A Power Efficient Neural Network Implementation on Heterogeneous FPGA and GPU Devices

Utilizing cloud FPGAs towards the open neural network standard

FPGA-based implementation of deep neural network using stochastic computing

A Compilation Toolchain of Neural Networks for FPGA Backend.

Deep neural network accelerator based on FPGA

A General Neural Network Hardware Architecture on FPGA

Spike Trains Encoding Optimization for Spiking Neural Networks Implementation in FPGA

A Design Methodology for Efficient Implementation of Deconvolutional Neural Networks on an FPGA

A neural network accelerated optimization method for FPGA

An FPGA Accelerated Method for Training Feed-forward Neural Networks Using Alternating Direction Method of Multipliers and LSMR

Towards Power Efficient DNN Accelerator Design on Reconfigurable Platform

CNN2Gate: Toward Designing a General Framework for Implementation of Convolutional Neural Networks on FPGA

Optimizing Neural Networks for Efficient FPGA Implementation: A Survey

A Survey of FPGA-Based Neural Network Accelerator

Accelerating Neural-ODE Inference on FPGAs with Two-Stage Structured Pruning and History-based Stepsize Search.

GaBAN: A Generic and Flexibly Programmable Vector Neuro-processor on FPGA

Performance analysis of multiple input single layer neural network hardware chip