Abstract:There is an increasing demand for technology and solutions to counter commercial, off-the-shelf small unmanned aerial systems (sUAS). Advances in machine learning and deep neural networks for object detection, coupled with lower cost and power requirements of cameras, led to promising vision-based solutions for sUAS detection. However, solely relying on the visible spectrum has previously led to reliability issues in low contrast scenarios such as sUAS flying below the treeline and against bright sources of light. Alternatively, due to the relatively high heat signatures emitted from sUAS during ight, a long-wave infrared (LWIR) sensor is able to produce images that clearly contrast the sUAS from its background. However, compared to widely available visible spectrum sensors, LWIR sensors have lower resolution and may produce more false positives when exposed to birds or other heat sources. This research work proposes combining the advantages of the LWIR and visible spectrum sensors using machine learning for vision-based detection of sUAS. Utilizing the heightened background contrast from the LWIR sensor combined and synchronized with the relatively increased resolution of the visible spectrum sensor, a deep learning model was trained to detect the sUAS through previously difficult environments. More specifically, the approach demonstrated effective detection of multiple sUAS flying above and below the treeline, in the presence of heat sources, and glare from the sun. Our approach achieved a detection rate of 71.2 8.3%, improving by 69% when compared to LWIR and by 30.4% when visible spectrum alone, and achieved false alarm rate of 2.7 2.6%, decreasing by 74.1% and by 47.1% when compared to LWIR and visible spectrum alone, respectively, on average, for single and multiple drone scenarios, controlled for the same confidence metric of the machine learning object detector of at least 50%. With a network of these small and affordable sensors, one can accurately estimate the 3D position of the sUAS, which could then be used for elimination or further localization from more narrow sensors, like a fire-control radar (FCR). Videos of the solution's performance can be seen at https://sites.google.com/view/tamudrone-spie2020/.

Multi-source sensor fusion for small unmanned aircraft systems using fuzzy logic

Position Estimation Method for Small Drones Based on the Fusion of Multisource, Multimodal Data and Digital Twins

Source Seeking Of Multi-Uav With Obstacle Avoidance

Radar/Visual Fusion With Fuse-Before-Track Strategy For Low Altitude Non-Cooperative Sense And Avoid

Unmanned Visual Localization Based on Satellite and Image Fusion

UAV Detection Multi-sensor Data Fusion

Simultaneous Localization and Mapping (SLAM) and Data Fusion in Unmanned Aerial Vehicles: Recent Advances and Challenges

Research on the Intelligent Obstacle Avoidance and Path Planning Strategy of UAV based on Multi-Sensor Fusion

Relative Positioning Method for UAVs Based on Multi-Source Information Fusion

Research on Multi-source Information Fusion based on Extended Kalman Filter

To Fuse or Not to Fuse: Measuring Consistency in Multi-Sensor Fusion for Aerial Robots

Multi-Stage Fusion Architecture for Small-Drone Localization and Identification Using Passive RF and EO Imagery: A Case Study

Multi-sensor integrated navigation/positioning systems using data fusion: From analytics-based to learning-based approaches

Mission-driven Autonomous Perception and Fusion Based on UAV Swarm

Multi-sensor Data Fusion Using Adaptive Kalman Filter

GOMSF: Graph-Optimization Based Multi-Sensor Fusion for robust UAV Pose estimation

Research on Robust Detection System of Vehicle Attitude Fusion Features Based on Multi-Sensor Fusion

Integrated Sensing and Communication enabled Multiple Base Stations Cooperative UAV Detection

Combining visible and infrared spectrum imagery using machine learning for small unmanned aerial system detection

Real-Time Onboard 3D State Estimation of an Unmanned Aerial Vehicle in Multi-Environments Using Multi-Sensor Data Fusion

Real-Time Multi-Modal Semantic Fusion on Unmanned Aerial Vehicles