Abstract:Recent years have seen a significant rise in gigapixel-level image/video capture systems and benchmarks with high-resolution wide (HRW) shots. Different from close-up shots like MS COCO, the higher resolution and wider field of view raise new research and application problems, such as how to perform accurate and efficient object detection with such large input in low-power edge devices like UAVs. There are several unique challenges in HRW shots. (1) Sparse information: the objects of interest cover less area. (2) Various scale: there is 10 to 100x x object scale change in one single image. (3) Incomplete objects: the sliding window strategy to handle the large input leads to truncated objects at the window edge. (4) Multi-scale information: it is unclear how to use multi-scale information in training and inference. Consequently, directly using a close-up detector leads to inaccuracy and inefficiency. In this paper, we systematically investigate this problem and bridge the gap between object detection in close-up and HRW shots, by introducing a novel sparse architecture that can be integrated with common networks like ConvNet and Transformer. It leverages alternative sparse learning to complementarily fuse coarse-grained and fine-grained features to (1) adaptively extract valuable information from (2) different object scales. We also propose a novel Cross-window Non-Maximum Suppression (C-NMS) algorithm to (3) improve the box merge from different windows. Furthermore, we propose a (4) simple yet effective multi-scale training and inference strategy to improve accuracy. Experiments on two benchmarks with HRW shots, PANDA and DOTA-v1.0, demonstrate that our methods significantly improve accuracy (up to 5.8%) and speed (up to 3x) over SotAs, for both ConvNet or Transformer based detectors, on edge devices. Our code is open-sourced and available at https://github.com/liwenxi/SparseFormer.

SDPDet: Learning Scale-Separated Dynamic Proposals for End-to-End Drone-View Detection

DroneNet: Rescue Drone-View Object Detection

A Lightweight SE-YOLOv3 Network for Multi-Scale Object Detection in Remote Sensing Imagery.

Real-Time Drone Detection Using Deep Learning Approach.

Scale Decoupled Pyramid for Object Detection in Aerial Images

SDSDet: A real-time object detector for small, dense, multi-scale remote sensing objects

Hybrid receptive field network for small object detection on drone view

DFS-DETR: Detailed-Feature-Sensitive Detector for Small Object Detection in Aerial Images Using Transformer

Scale Enhancement Pyramid Network for Small Object Detection from UAV Images

SODCNN: A Convolutional Neural Network Model for Small Object Detection in Drone-Captured Images

Detection Based on Semantics and a Detail Infusion Feature Pyramid Network and a Coordinate Adaptive Spatial Feature Fusion Mechanism Remote Sensing Small Object Detector

CSSDet: small object detection via cross-scale feature enhancement on drone-view images

Scale-Adaptive Salience Supervision and Dynamic Token Filtering for Small Object Detection in Remote Sensing Images

ZoomInNet: A Novel Small Object Detector in Drone Images with Cross-Scale Knowledge Distillation

Drone-DETR: Efficient Small Object Detection for Remote Sensing Image Using Enhanced RT-DETR Model

Hierarchical Scale Awareness for Object Detection in Unmanned Aerial Vehicle Scenes

Bridging the Gap Between Object Detection in Close-Up and High-Resolution Wide Shots

MFRENet: efficient detection of drone image based on multiscale feature aggregation and receptive field expanded

SliNet: Slicing-Aided Learning for Small Object Detection

ESOD: Efficient Small Object Detection on High-Resolution Images

QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection