Abstract:Deep learning and Convolutional Neural Network (CNN) have becoming increasingly more popular and important in both academic and industrial areas in recent years cause they are able to provide better accuracy and result in classification, detection and recognition areas, compared to traditional approaches. Currently, there are many popular frameworks in the market for deep learning development, such as Caffe, TensorFlow, Pytorch, and most of frameworks natively support CPU and consider GPU as the mainline accelerator by default. FPGA device, viewed as a potential heterogeneous platform, still cannot provide a comprehensive support for CNN development in popular frameworks, in particular to the training phase. In this paper, we firstly propose the FeCaffe, i.e. FPGA-enabled Caffe, a hierarchical software and hardware design methodology based on the Caffe to enable FPGA to support mainline deep learning development features, e.g. training and inference with Caffe. Furthermore, we provide some benchmarks with FeCaffe by taking some classical CNN networks as examples, and further analysis of kernel execution time in details accordingly. Finally, some optimization directions including FPGA kernel design, system pipeline, network architecture, user case application and heterogeneous platform levels, have been proposed gradually to improve FeCaffe performance and efficiency. The result demonstrates the proposed FeCaffe is capable of supporting almost full features during CNN network training and inference respectively with high degree of design flexibility, expansibility and reusability for deep learning development. Compared to prior studies, our architecture can support more network and training settings, and current configuration can achieve 6.4x and 8.4x average execution time improvement for forward and backward respectively for LeNet.

A Deep Learning Frame on Embedded Multicore Processors Based on Caffe and Its Parallel Implementation

DaDianNao: A Machine-Learning Supercomputer

Performance Evaluation Of Deep Learning Frameworks On Embedded Gpu

FeCaffe: FPGA-enabled Caffe with OpenCL for Deep Learning Training and Inference on Intel Stratix 10

OpenCL caffe: Accelerating and enabling a cross platform machine learning framework

A deep learning image recognition framework accelerator based parallel computing

An Efficient Parallel CNN Inference Framework for Multi-zone Processor

A Manycore Processor Based Multilayer Perceptron Feedforward Acceleration Framework for Embedded System

Cross Hardware-Software Boundary Exploration for Scalable and Optimized Deep Learning Platform Design

CAP: Communication-aware Automated Parallelization for Deep Learning Inference on CMP Architectures

SingleCaffe: an Efficient Framework for Deep Learning on a Single Node

CoDL: efficient CPU-GPU co-execution for deep learning inference on mobile devices

An Embedded Inference Framework for Convolutional Neural Network Applications

swCaffe: a Parallel Framework for Accelerating Deep Learning Applications on Sunway TaihuLight

Exploiting Potential of Deep Neural Networks by Layer-Wise Fine-Grained Parallelism

Deep Learning Inference on Heterogeneous Mobile Processors: Potentials and Pitfalls

Design and Implementation of Deep Neural Network for Edge Computing

A Parallel Processing CNN Accelerator on Embedded Devices Based on Optimized MobileNet

A Scalable Object Detection Framework Based on Embedded Manycore Cluster

Optimized Parallel Implementation of Face Detection Based on Embedded Heterogeneous Many-Core Architecture

Parallelizing Convolutional Neural Networks On Intel (R) Many Integrated Core Architecture