Abstract:Deep neural networks (DNNs), as the basis of object detection, will play a key role in the development of future autonomous systems with full autonomy. The autonomous systems have special requirements of real-time, energy-e cient implementations of DNNs on a power-budgeted system. Two research thrusts are dedicated to per- formance and energy e ciency enhancement of the inference phase of DNNs. The first one is model compression techniques while the second is e cient hardware implementations. Recent researches on extremely-low-bit CNNs such as binary neural network (BNN) and XNOR-Net replace the traditional oating point operations with bi- nary bit operations, signi cantly reducing memory bandwidth and storage requirement, whereas suffering non-negligible accuracy loss and waste of digital signal processing (DSP) blocks on FPGAs. To overcome these limitations, this paper proposes REQ-YOLO, a resource aware, systematic weight quantization framework for object detection, considering both algorithm and hardware resource aspects in object detection. We adopt the block-circulant matrix method and propose a heterogeneous weight quantization using Alternative Direction Method of Multipliers (ADMM), an e ective optimization technique for general, non-convex optimization problems. To achieve real-time, highly efficient implementations on FPGA, we present the detailed hardware implementation of block circulant matrices on CONV layers and de- velop an e cient processing element (PE) structure supporting the heterogeneous weight quantization, CONV data ow and pipelining techniques, design optimization, and a template-based automatic synthesis framework to optimally exploit hardware resource. Experimental results show that our proposed REQ-YOLO framework can signi cantly compress the YOLO model while introducing very small accuracy degradation. The related codes are here: https://github.com/Anonymous788/heterogeneous_ADMM_YOLO.

Lightweight Convolutional Neural Network of YOLO V3- Tiny Algorithm on FPGA for Target Detection

High-performance Reconfigurable DNN Accelerator on a Bandwidth-limited Embedded System

Reduced-Parameter YOLO-Like Object Detector Oriented to Resource-Constrained Platform

LPYOLO: Low Precision YOLO for Face Detection on FPGA

A Low-Latency FPGA Implementation for Real-Time Object Detection

Design and Implementation of YOLOv3-Tiny Accelerator Based on PYNQ-Z2 Heterogeneous Platform

A dedicated hardware accelerator for real-time acceleration of YOLOv2

Design and implementation of FPGA-based deep learning object detection system

FPGA-Based Vehicle Detection and Tracking Accelerator

FPGA-Based Real-Time Object Detection and Classification System Using YOLO for Edge Computing

WGeod: A General and Efficient FPGA Accelerator for Object Detection

Hardware Acceleration for Object Detection using YOLOv5 Deep Learning Algorithm on Xilinx Zynq FPGA Platform

An FPGA-Based YOLOv5 Accelerator for Real-Time Industrial Vision Applications

Edge Real-Time Object Detection and DPU-Based Hardware Implementation for Optical Remote Sensing Images

An FPGA-Based Reconfigurable CNN Accelerator for YOLO

REQ-YOLO

Algorithm-Hardware Co-Design of Single Shot Detector for Fast Object Detection on FPGAs

A CNN Hardware Accelerator Designed for YOLO Algorithm Based on RISC-V SoC

An Efficient Real-Time Object Detection Framework on Resource-Constricted Hardware Devices via Software and Hardware Co-design

Object Detection Edge Performance Optimization on FPGA-Based Heterogeneous Multiprocessor Systems

A Lightweight Detection Method for Remote Sensing Images and Its Energy-Efficient Accelerator on Edge Devices