Abstract:Usage of Unmanned Aerial Vehicles (UAVs) is growing rapidly in a wide range of consumer applications, as they prove to be both autonomous and flexible in a variety of environments and tasks. However, this versatility and ease of use also brings a rapid evolution of threats by malicious actors that can use UAVs for criminal activities, converting them to passive or active threats. The need to protect critical infrastructures and important events from such threats has brought advances in counter UAV (c-UAV) applications. Nowadays, c-UAV applications offer systems that comprise a multi-sensory arsenal often including electro-optical, thermal, acoustic, radar and radio frequency sensors, whose information can be fused to increase the confidence of threat’s identification. Nevertheless, real-time surveillance is a cumbersome process, but it is absolutely essential to detect promptly the occurrence of adverse events or conditions. To that end, many challenging tasks arise such as object detection, classification, multi-object tracking and multi-sensor information fusion. In recent years, researchers have utilized deep learning based methodologies to tackle these tasks for generic objects and made noteworthy progress, yet applying deep learning for UAV detection and classification is considered a novel concept. Therefore, the need to present a complete overview of deep learning technologies applied to c-UAV related tasks on multi-sensor data has emerged. The aim of this paper is to describe deep learning advances on c-UAV related tasks when applied to data originating from many different sensors as well as multi-sensor information fusion. This survey may help in making recommendations and improvements of c-UAV applications for the future.

Deep Learning Computer Vision Algorithms for Real-time UAVs On-board Camera Image Processing

Fast Detection and Recognition Method of UAV in Sky Background

Design and Implementation of a UAV-Based Airborne Computing Platform for Computer Vision and Machine Learning Applications

Using deep learning in an embedded system for real-time target detection based on images from an unmanned aerial vehicle: vehicle detection as a case study

Deep Neural Network Architecture Search for Accurate Visual Pose Estimation aboard Nano-UAVs

Testbed for Connected Artificial Intelligence using Unmanned Aerial Vehicles and Convolutional Pose Machines

An Efficient UAV-based Artificial Intelligence Framework for Real-Time Visual Tasks

Event-driven Vision and Control for UAVs on a Neuromorphic Chip

AERO: AI-Enabled Remote Sensing Observation with Onboard Edge Computing in UAVs

Real-Time Multi-Modal Active Vision for Object Detection on UAVs Equipped With Limited Field of View LiDAR and Camera

CNN-based Human Detection for UAVs in Search and Rescue

Efficient Object Detection Model for Real-Time UAV Applications

Real-Time Cross-View Image Matching and Camera Pose Determination for Unmanned Aerial Vehicles

Implementation of an Edge-Computing Vision System on Reduced-Board Computers Embedded in UAVs for Intelligent Traffic Management

Enhancing UAV Visual Landing Recognition with YOLO's Object Detection by Onboard Edge Computing

Deep Learning on Multi Sensor Data for Counter UAV Applications—A Systematic Review

Deep Learning-Based Real-Time Multiple-Object Detection and Tracking from Aerial Imagery via a Flying Robot with GPU-Based Embedded Devices

A fast, lightweight deep learning vision pipeline for autonomous UAV landing support with added robustness

Real-time Aerial Detection and Reasoning on Embedded-UAVs

An Open Source and Open Hardware Deep Learning-powered Visual Navigation Engine for Autonomous Nano-UAVs