Abstract:There is an increasing demand for technology and solutions to counter commercial, off-the-shelf small unmanned aerial systems (sUAS). Advances in machine learning and deep neural networks for object detection, coupled with lower cost and power requirements of cameras, led to promising vision-based solutions for sUAS detection. However, solely relying on the visible spectrum has previously led to reliability issues in low contrast scenarios such as sUAS flying below the treeline and against bright sources of light. Alternatively, due to the relatively high heat signatures emitted from sUAS during ight, a long-wave infrared (LWIR) sensor is able to produce images that clearly contrast the sUAS from its background. However, compared to widely available visible spectrum sensors, LWIR sensors have lower resolution and may produce more false positives when exposed to birds or other heat sources. This research work proposes combining the advantages of the LWIR and visible spectrum sensors using machine learning for vision-based detection of sUAS. Utilizing the heightened background contrast from the LWIR sensor combined and synchronized with the relatively increased resolution of the visible spectrum sensor, a deep learning model was trained to detect the sUAS through previously difficult environments. More specifically, the approach demonstrated effective detection of multiple sUAS flying above and below the treeline, in the presence of heat sources, and glare from the sun. Our approach achieved a detection rate of 71.2 8.3%, improving by 69% when compared to LWIR and by 30.4% when visible spectrum alone, and achieved false alarm rate of 2.7 2.6%, decreasing by 74.1% and by 47.1% when compared to LWIR and visible spectrum alone, respectively, on average, for single and multiple drone scenarios, controlled for the same confidence metric of the machine learning object detector of at least 50%. With a network of these small and affordable sensors, one can accurately estimate the 3D position of the sUAS, which could then be used for elimination or further localization from more narrow sensors, like a fire-control radar (FCR). Videos of the solution's performance can be seen at https://sites.google.com/view/tamudrone-spie2020/.

Broad Area Search and Detection of Surface-to-Air Missile Sites Using Spatial Fusion of Component Object Detections from Deep Neural Networks

ObjectFusion: an Object Detection and Segmentation Framework with RGB-D SLAM and Convolutional Neural Networks

Object Detection in Aerial Images Using Feature Fusion Deep Networks

Airplane Detection Based on Fusion Framework by Combining Saliency Model with Deep Convolutional Neural Networks.

An Adaptive Infrared Small-Target-Detection Fusion Algorithm Based on Multiscale Local Gradient Contrast for Remote Sensing

A Feature Fusion Deep-Projection Convolution Neural Network for Vehicle Detection in Aerial Images

Airborne Small Target Detection Method Based on Multimodal and Adaptive Feature Fusion

Channelwise and Spatially Guided Multimodal Feature Fusion Network for 3-D Object Detection in Autonomous Vehicles

Fusion Object Detection of Satellite Imagery with Arbitrary-Oriented Region Convolutional Neural Network

Combining visible and infrared spectrum imagery using machine learning for small unmanned aerial system detection

Small object detection model for remote sensing images combining super-resolution assisted reasoning and dynamic feature fusion

Adaptive Feature Fusion With Attention-Guided Small Target Detection in Remote Sensing Images

Multi-Channel CNN-based Object Detection for Enhanced Situation Awareness

Spatial Information Enhancement with Multi-Scale Feature Aggregation for Long-Range Object and Small Reflective Area Object Detection from Point Cloud

Multiscale Progressive Fusion Filter Network for Infrared Small Target Detection

Structured Object-Level Relational Reasoning CNN-Based Target Detection Algorithm in a Remote Sensing Image

A Novel Wide-Area Multiobject Detection System with High-Probability Region Searching

Multimodal Fusion Object Detection System for Autonomous Vehicles

Research on Detection and Recognition Technology of a Visible and Infrared Dim and Small Target Based on Deep Learning

Multi-scale direction-aware SAR object detection network via global information fusion

Multi-Scale Geospatial Object Detection Based on Shallow-Deep Feature Extraction