Abstract:RGBT person detection benefits numerous vital applications like surveillance, search, and rescue. Meanwhile, drones can capture images holding broad perspectives and large searching regions per frame, which can notably improve the efficacy of large-scale search and rescue missions. In this work, we leverage the advantages of drone-based vision for RGBT person detection. The drone-based RGBT person detection task brings interesting challenges to existing cross-modality object detectors, e.g., tiny sizes of objects, modality-space imbalance, and position shifts. Observing that there is a lack of data and customized detectors for drone-based RGBT person detection, we contribute two new datasets and design a novel detector. The data contribution is two-fold. For one, we construct the first large-scale drone-based RGBT person detection benchmark RGBTDronePerson, which contains 6,125 pairs of RGBT images and 70,880 instances. Images are captured in various scenes and under various illumination and weather conditions. For another, we annotate the VTUAV tracking dataset and obtain its object detection version, named VTUAV-det. To tackle the challenges raised by this task, we propose a Quality-aware RGBT Fusion Detector (QFDet). Firstly, we design a Quality-aware Learning Strategy (QLS) to provide sufficient supervision for tiny objects while focusing on high-quality samples, in which a Quality-Aware Factor (QAF) is designed to measure the quality. Moreover, a Quality-aware Cross-modality Enhancement module (QCE) is proposed to predict a QAF map for each modality, which not only indicates the reliability of each modality but also highlights regions where objects are more likely to appear. Our QFDet remarkably boosts the detection performance over tiny and small objects, surpassing the strong baseline on mAP 50 tiny by 6.57 points on RGBTDronePerson and mAP s by 3.80 points on VTUAV-det. The datasets, codes, and pre-trained models are available at https://nnnnerd.github.io/RGBTDronePerson/.

Drone-Person Tracking in Uniform Appearance Crowd: A New Dataset

The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking

Robot Person Following in Uniform Crowd Environment

DenseTrack: Drone-based Crowd Tracking via Density-aware Motion-appearance Synergy

Object-Level Pseudo-3D Lifting for Distance-Aware Tracking

UAVSwarm Dataset: An Unmanned Aerial Vehicle Swarm Dataset for Multiple Object Tracking

Online Single Person Tracking For Unmanned Aerial Vehicles: Benchmark And New Baseline

The P-DESTRE: A Fully Annotated Dataset for Pedestrian Detection, Tracking, Re-Identification and Search from Aerial Devices

Aerial Tracking of Camouflaged People in Woodlands.

OLOD: a new UAV dataset and benchmark for single tiny object tracking

VisDrone-CC2020: The Vision Meets Drone Crowd Counting Challenge Results

Drone-type-Set: Drone types detection benchmark for drone detection and tracking

SFTrack: A Robust Scale and Motion Adaptive Algorithm for Tracking Small and Fast Moving Objects

Multi-Drone-Based Single Object Tracking With Agent Sharing Network

Tracking Pedestrian Heads in Dense Crowd

Object Tracking in Unmanned Aerial Vehicle Videos via Multifeature Discrimination and Instance-Aware Attention Network

SOMPT22: A Surveillance Oriented Multi-Pedestrian Tracking Dataset

Drone-based RGBT tiny person detection

Camouflaged Object Tracking: A Benchmark

Vision-Based Anti-UAV Detection and Tracking

MMPTRACK: Large-scale Densely Annotated Multi-camera Multiple People Tracking Benchmark