Abstract:There is an increasing demand for technology and solutions to counter commercial, off-the-shelf small unmanned aerial systems (sUAS). Advances in machine learning and deep neural networks for object detection, coupled with lower cost and power requirements of cameras, led to promising vision-based solutions for sUAS detection. However, solely relying on the visible spectrum has previously led to reliability issues in low contrast scenarios such as sUAS flying below the treeline and against bright sources of light. Alternatively, due to the relatively high heat signatures emitted from sUAS during ight, a long-wave infrared (LWIR) sensor is able to produce images that clearly contrast the sUAS from its background. However, compared to widely available visible spectrum sensors, LWIR sensors have lower resolution and may produce more false positives when exposed to birds or other heat sources. This research work proposes combining the advantages of the LWIR and visible spectrum sensors using machine learning for vision-based detection of sUAS. Utilizing the heightened background contrast from the LWIR sensor combined and synchronized with the relatively increased resolution of the visible spectrum sensor, a deep learning model was trained to detect the sUAS through previously difficult environments. More specifically, the approach demonstrated effective detection of multiple sUAS flying above and below the treeline, in the presence of heat sources, and glare from the sun. Our approach achieved a detection rate of 71.2 8.3%, improving by 69% when compared to LWIR and by 30.4% when visible spectrum alone, and achieved false alarm rate of 2.7 2.6%, decreasing by 74.1% and by 47.1% when compared to LWIR and visible spectrum alone, respectively, on average, for single and multiple drone scenarios, controlled for the same confidence metric of the machine learning object detector of at least 50%. With a network of these small and affordable sensors, one can accurately estimate the 3D position of the sUAS, which could then be used for elimination or further localization from more narrow sensors, like a fire-control radar (FCR). Videos of the solution's performance can be seen at https://sites.google.com/view/tamudrone-spie2020/.

Low-Rank Approximation and Multiple Sparse Constraint Modeling for Infrared Low-Flying Fixed-Wing UAV Detection

Low in Resolution, High in Precision: UAV Detection with Super-Resolution and Motion Information Extraction

Fast Detection and Recognition Method of UAV in Sky Background

Sparse-representation-based automatic target detection ininfrared imagery

Real-Time Recognition Algorithm of Small Target for UAV Infrared Detection

5-D Spatial-Temporal Information-Based Infrared Small Target Detection in Complex Environments

Non-Convex Tensor Low-Rank Approximation for Infrared Small Target Detection

IMNN-LWEC: A Novel Infrared Small Target Detection Based on Spatial–Temporal Tensor Model

Learning Nonlocal Quadrature Contrast for Detection and Recognition of Infrared Rotary-Wing UAV Targets in Complex Background

Precision Detection of Infrared Small Target in Ground-to-Air Scene

Robust infrared small target detection via non-negativity constraint-based sparse representation

Infrared Small Target Detection with Patch Tensor Collaborative Sparse and Total Variation Constraint

Representative Coefficient Total Variation for Efficient Infrared Small Target Detection

Single-Frame Infrared Small Target Detection by High Local Variance, Low-Rank and Sparse Decomposition

UAV Target Detection under Complex Sky Background

A Two-Layer Detection Model for Infrared Slow Low-Altitude Targets

A Novel Infrared Small Target Detection Algorithm Based on Deep Learning

Extended Infrared Target Filtering via Random Finite Set and Low‐Rank Matrix Decomposition

ANLPT: Self-Adaptive and Non-Local Patch-Tensor Model for Infrared Small Target Detection

High-precision real-time UAV target recognition based on improved YOLOv4

Combining visible and infrared spectrum imagery using machine learning for small unmanned aerial system detection