Abstract:Recent years have seen a significant rise in gigapixel-level image/video capture systems and benchmarks with high-resolution wide (HRW) shots. Different from close-up shots like MS COCO, the higher resolution and wider field of view raise new research and application problems, such as how to perform accurate and efficient object detection with such large input in low-power edge devices like UAVs. There are several unique challenges in HRW shots. (1) Sparse information: the objects of interest cover less area. (2) Various scale: there is 10 to 100x x object scale change in one single image. (3) Incomplete objects: the sliding window strategy to handle the large input leads to truncated objects at the window edge. (4) Multi-scale information: it is unclear how to use multi-scale information in training and inference. Consequently, directly using a close-up detector leads to inaccuracy and inefficiency. In this paper, we systematically investigate this problem and bridge the gap between object detection in close-up and HRW shots, by introducing a novel sparse architecture that can be integrated with common networks like ConvNet and Transformer. It leverages alternative sparse learning to complementarily fuse coarse-grained and fine-grained features to (1) adaptively extract valuable information from (2) different object scales. We also propose a novel Cross-window Non-Maximum Suppression (C-NMS) algorithm to (3) improve the box merge from different windows. Furthermore, we propose a (4) simple yet effective multi-scale training and inference strategy to improve accuracy. Experiments on two benchmarks with HRW shots, PANDA and DOTA-v1.0, demonstrate that our methods significantly improve accuracy (up to 5.8%) and speed (up to 3x) over SotAs, for both ConvNet or Transformer based detectors, on edge devices. Our code is open-sourced and available at https://github.com/liwenxi/SparseFormer.

Towards real-time object detection in GigaPixel-level video

Multilevel Spatial-Temporal Feature Aggregation for Video Object Detection

Compressive Sensing Ghost Imaging Object Detection Using Generative Adversarial Networks

Adaptive Optimization Strategies for Gigapixel Object Detection.

SaccadeDet: A Novel Dual-Stage Architecture for Rapid and Accurate Detection in Gigapixel Images

GigaHumanDet: Exploring Full-Body Detection on Gigapixel-Level Images

Multi-view Aggregation for Real-Time Accurate Object Detection of a Moving Camera

Accelerating real‐time object detection in high‐resolution video surveillance

SaccadeMOT: Enhancing Object Detection and Tracking in Gigapixel Images Via Scale-Aware Density Estimation

SDSDet: A real-time object detector for small, dense, multi-scale remote sensing objects

Adaptive Scale and Spatial Aggregation for Real-Time Object Detection

Real-Time and Accurate Object Detection in Compressed Video by Long Short-term Feature Aggregation

Real-time Object Detection Towards High Power Efficiency.

Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detection

Towards Real-Time Object Detection on Embedded Systems.

Bridging the Gap Between Object Detection in Close-Up and High-Resolution Wide Shots

PVGNet: A Bottom-Up One-Stage 3D Object Detector with Integrated Multi-Level Features

SG-Det: Shuffle-GhostNet-Based Detector for Real-Time Maritime Object Detection in UAV Images

QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection

Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection from Point Clouds

CA2Det: Cascaded Adaptive Fusion Pyramid Network Based on Attention Mechanism for Small Object Detection