Abstract:Recent years have seen a significant rise in gigapixel-level image/video capture systems and benchmarks with high-resolution wide (HRW) shots. Different from close-up shots like MS COCO, the higher resolution and wider field of view raise new research and application problems, such as how to perform accurate and efficient object detection with such large input in low-power edge devices like UAVs. There are several unique challenges in HRW shots. (1) Sparse information: the objects of interest cover less area. (2) Various scale: there is 10 to 100x x object scale change in one single image. (3) Incomplete objects: the sliding window strategy to handle the large input leads to truncated objects at the window edge. (4) Multi-scale information: it is unclear how to use multi-scale information in training and inference. Consequently, directly using a close-up detector leads to inaccuracy and inefficiency. In this paper, we systematically investigate this problem and bridge the gap between object detection in close-up and HRW shots, by introducing a novel sparse architecture that can be integrated with common networks like ConvNet and Transformer. It leverages alternative sparse learning to complementarily fuse coarse-grained and fine-grained features to (1) adaptively extract valuable information from (2) different object scales. We also propose a novel Cross-window Non-Maximum Suppression (C-NMS) algorithm to (3) improve the box merge from different windows. Furthermore, we propose a (4) simple yet effective multi-scale training and inference strategy to improve accuracy. Experiments on two benchmarks with HRW shots, PANDA and DOTA-v1.0, demonstrate that our methods significantly improve accuracy (up to 5.8%) and speed (up to 3x) over SotAs, for both ConvNet or Transformer based detectors, on edge devices. Our code is open-sourced and available at https://github.com/liwenxi/SparseFormer.

Where-What Network With Cuda: General Object Recognition And Location In Complex Backgrounds

A biologically-inspired network for generic object recognition using CUDA

Real-time Semantic Segmentation with Weighted Factorized-Depthwise Convolution

Where-What Network 5: Dealing with Scales for Objects in Complex Backgrounds

Synapse Maintenance in the Where-What Networks

WW-Nets: Dual Neural Networks for Object Detection

GCWNet: A Global Context-Weaving Network for Object Detection in Remote Sensing Images

Spatially-Aware Context Neural Networks.

The Architecture and Recognition Algorithm in Haibao Perceptual Development Robot

Skull-closed Autonomous Development : WWN-7 Dealing with Scales

Skull-closed Autonomous Development: WWN-6 Using Natural Video.

Bridging the Gap Between Object Detection in Close-Up and High-Resolution Wide Shots

A Two-Pipeline Instance Segmentation Network via Boundary Enhancement for Scene Understanding

An effective head detection framework via convolutional neural networks

Object Detection Networks on Convolutional Feature Maps

WGNet: Wider Graph Convolution Networks for 3D Point Cloud Classification with Local Dilated Connecting and Context-Aware

AGO-Net: Association-Guided 3D Point Cloud Object Detection Network

High-speed railway real-time localization auxiliary method based on deep neural network

Real-time Interactive Object Extraction System for High Resolution Remote Sensing Images Based on Parallel Computing Architecture

C2BG-Net: Cross-modality and cross-scale balance network with global semantics for multi-modal 3D object detection

W-Net: Convolutional neural network for segmenting remote sensing images by dual path semantics