RTM-UAVDet: A Real-Time Multimodal UAV Detector

Guorui Wang,Qian Jiang,Xin Jin,Michal Wozniak,Puming Wang Shaowen Yao
DOI: https://doi.org/10.1109/TAES.2024.3446460
IF: 3.491
2024-01-01
IEEE Transactions on Aerospace and Electronic Systems
Abstract:As unmanned aerial vehicles (UAVs) become more prevalent, the need for accurate and reliable detection algorithms becomes increasingly important. UAV detection algorithms are essential for maintaining security, managing airspace, safeguarding privacy, responding to emergencies, protecting critical infrastructure, ensuring regulatory compliance, and enabling responsible UAV usage across various domains. The multi-scene images and videos based on visible light (RGB) and thermal infrared (TIR) remote sensing of UAVs are considered crucial data sources for public safety. However, as imaging techniques move from single to multimodality, using object detection algorithms to detect RGB and TIR modalities simultaneously in real-time is a significant challenge. This study proposes a multimodal UAV object detection framework for static images. Based on convolutional neural network (CNN) architecture, the multimodal dynamic convolution model extracts features from ground TIR images and videos of forward-looking infrared (FLIR) and visible light cameras. Our approach circumvents the constraints of conventional multimodal fusion methods by effectively detecting UAVs within single-frame, multimodal static images without requiring image alignment. The results show that the average precision 0.5:0.95 ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$AP_{0.5:0.95}$</tex-math></inline-formula> ) of the UAV instance in the validation task is 66.1%. Furthermore, RTM-UAVDet operates at a speed of 72.4 frames per second (FPS), fully satisfying the requirements for real-time processing and effectively recognizing UAVs larger than 48 (6×8) pixels.
What problem does this paper attempt to address?