Abstract:As unmanned aerial vehicles (UAVs) become more prevalent, the need for accurate and reliable detection algorithms becomes increasingly important. UAV detection algorithms are essential for maintaining security, managing airspace, safeguarding privacy, responding to emergencies, protecting critical infrastructure, ensuring regulatory compliance, and enabling responsible UAV usage across various domains. The multi-scene images and videos based on visible light (RGB) and thermal infrared (TIR) remote sensing of UAVs are considered crucial data sources for public safety. However, as imaging techniques move from single to multimodality, using object detection algorithms to detect RGB and TIR modalities simultaneously in real-time is a significant challenge. This study proposes a multimodal UAV object detection framework for static images. Based on convolutional neural network (CNN) architecture, the multimodal dynamic convolution model extracts features from ground TIR images and videos of forward-looking infrared (FLIR) and visible light cameras. Our approach circumvents the constraints of conventional multimodal fusion methods by effectively detecting UAVs within single-frame, multimodal static images without requiring image alignment. The results show that the average precision 0.5:0.95 ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$AP_{0.5:0.95}$</tex-math></inline-formula> ) of the UAV instance in the validation task is 66.1%. Furthermore, RTM-UAVDet operates at a speed of 72.4 frames per second (FPS), fully satisfying the requirements for real-time processing and effectively recognizing UAVs larger than 48 (6×8) pixels.

UAV Target Detection under Complex Sky Background

Fast Detection and Recognition Method of UAV in Sky Background

Low in Resolution, High in Precision: UAV Detection with Super-Resolution and Motion Information Extraction

Detection of Military Targets on Ground and Sea by UAVs with Low-Altitude Oblique Perspective

Target detection approach for UAV via Hybrid Model

Research on Target Location of UAV Video

UAV Target Identification Based on Information fusion of Computer Vision and Radar Signals

Low-altitude protection technology of anti-UAVs based on multisource detection information fusion

A Vision-based UAV Tracker Aiming at Aerial Targets

Realizing Small UAV Targets Recognition via Multi-Dimensional Feature Fusion of High-Resolution Radar

A Smart Radar Signal Processing Solution for Ground-Based UAVs Surveillance.

Research on Information Fusion of Computer Vision and Radar Signals in UAV Target Identification

Research on UAV Obstacle Detection Based on Data Fusion of Millimeter Wave Radar and Monocular Camera

Flying Small Target Detection For Anti-Uav Based On A Gaussian Mixture Model In A Compressive Sensing Domain

Deep Learning-Based Unmanned Aerial Vehicle Detection in the Low Altitude Clutter Background

A Low-Slow-Small UAV Detection Method Based on Fusion of Range Doppler Map and Satellite Map

Recognition of flying small target based on enhanced quadratic time-frequency analysis

RTM-UAVDet: A Real-Time Multimodal UAV Detector

UAV Detection Multi-sensor Data Fusion

A High-resolution Radar Automatic Target Recognition Method for Small UAVs Based on Multi-feature Fusion

A Detection and Information Extraction Approach for UAV Swarm