Abstract:Object detection via deep neural networks has undergone considerable advancements in recent years. Yet, the detection of smaller objects, specifically those with a few pixels (i.e., < 322 pixels), is still challenging compared with large objects (i.e., > 962 pixels). Existing methods commonly apply high-resolution features or complex super-resolution strategies based on the two-stage Faster Region Convolutional Neural Network (RCNN). They sequentially apply localization and classification stages after a shared feature map extracted by one single backbone network. However, these methods cause low detection accuracy of small objects, high computational overhead, and waste of hardware resources. In this paper, we develop a high-accuracy and real-time small object detection system with negligible computational overhead and low hardware idleness. At the software level, we propose a two-stage Coarse-to-Fine Decoupling RCNN (CFD RCNN) with three techniques: (1) The shared backbone decoupling for localization and classification to achieve high accuracy for both tasks; (2) The training method using backbone feature upsampling for localization with low computational overhead; (3) The object cropping strategy from the original high-resolution image for high-accuracy classification. At the hardware level, we propose a virtualized FPGA accelerator with the Dynamic Resource Allocation (DRA) strategy. The DRA strategy reallocates the hardware resources, considering the workload and resource preference of each stage in CFD RCNN to reduce hardware idleness. Extensive experiments on the TT100K and GTSDB datasets using Xilinx ZCU102 FPGA show that the proposed small object detection system can achieve 2.9% improvement in mean average precision (mAP) compared with state-of-the-art (SOTA) algorithms and raised the throughput from 18.9 FPS to > 26.0 FPS (~1.37×) compared with existing accelerators.

Performance comparison of CNN, QNN and BNN deep neural networks for real-time object detection using ZYNQ FPGA node

High-performance Reconfigurable DNN Accelerator on a Bandwidth-limited Embedded System

Towards High-accuracy and Real-time Two-stage Small Object Detection on FPGA

A Fully Onchip Binarized Convolutional Neural Network FPGA Impelmentation with Accurate Inference.

An FPGA Accelerator Design of Spiking Neural Network for Energy-Efficient Object Detection

An FPGA Implementation of Deep Spiking Neural Networks for Low-Power and Fast Classification

Spike Trains Encoding Optimization for Spiking Neural Networks Implementation in FPGA

Efficient Deployment of Single Shot Multibox Detector Network on FPGAs

High Power-Efficient and Performance-Density FPGA Accelerator for CNN-Based Object Detection

Accelerating Low Bit-Width Convolutional Neural Networks with Embedded FPGA.

Binarized Depthwise Separable Neural Network for Object Tracking in FPGA

Design and implementation of FPGA-based deep learning object detection system

A Low-Latency FPGA Implementation for Real-Time Object Detection

Design of Convolutional Neural Network Based on FPGA

Going Deeper with Embedded FPGA Platform for Convolutional Neural Network

ZynqNet: An FPGA-Accelerated Embedded Convolutional Neural Network

Design and Implementation of an FPGA-Based DNN Architecture for Real-time Outlier Detection

A Reconfigurable Neural Network Processor With Tile-Grained Multicore Pipeline for Object Detection on FPGA

Highly Efficient SNNs for High-speed Object Detection

FPGA Implementation of Quantized Convolutional Neural Networks.