Abstract:In the last twenty years, unmanned aerial vehicles (UAVs) have garnered growing interest due to their expanding applications in both military and civilian domains. Detecting non-cooperative aerial vehicles with efficiency and estimating collisions accurately are pivotal for achieving fully autonomous aircraft and facilitating Advanced Air Mobility (AAM). This paper presents a deep-learning framework that utilizes optical sensors for the detection, tracking, and distance estimation of non-cooperative aerial vehicles. In implementing this comprehensive sensing framework, the availability of depth information is essential for enabling autonomous aerial vehicles to perceive and navigate around obstacles. In this work, we propose a method for estimating the distance information of a detected aerial object in real time using only the input of a monocular camera. In order to train our deep learning components for the object detection, tracking and depth estimation tasks we utilize the Amazon Airborne Object Tracking (AOT) Dataset. In contrast to previous approaches that integrate the depth estimation module into the object detector, our method formulates the problem as image-to-image translation. We employ a separate lightweight encoder-decoder network for efficient and robust depth estimation. In a nutshell, the object detection module identifies and localizes obstacles, conveying this information to both the tracking module for monitoring obstacle movement and the depth estimation module for calculating distances. Our approach is evaluated on the Airborne Object Tracking (AOT) dataset which is the largest (to the best of our knowledge) air-to-air airborne object dataset.

UAV environmental perception and autonomous obstacle avoidance: A deep learning and depth camera combined solution

Monocular Depth Estimation for UAV Obstacle Avoidance

Obstacle visual sensing based on deep learning for low-altitude small unmanned aerial vehicles

UAV Visual Flight Control Method Based on Deep Reinforcement Learning

UGV autonomous driving system design for unstructed environment

Fusing Stereopsis & Corner Sparse Optical Flow for Real-time Obstacle Avoidance of Unmanned Aerial Vehicle

Autonomous aerial obstacle avoidance using LiDAR sensor fusion

End-to-End Learning-Based Obstacle Avoidance for Fixed-Wing UAVs

UAV Low-Altitude Obstacle Detection Based on the Fusion of LiDAR and Camera

A Deep Learning Method Based on the Multi-stage Fusion of Radar and Camera in UAV Obstacle Avoidance

Deep-learning based autonomous-exploration for UAV navigation

Ensuring UAV Safety: A Vision-only and Real-time Framework for Collision Avoidance Through Object Detection, Tracking, and Distance Estimation

Robust Autonomous Landing of UAVs in Non-Cooperative Environments Based on Comprehensive Terrain Understanding

Autonomous Obstacle Avoidance for UAV Based on Fusion of Radar and Monocular Camera

Real-Time Compact Environment Representation for UAV Navigation.

Toward Embodied Intelligence: An Autonomous Exploration System for Multimodal UAV

Robust Autonomous Landing of UAV in Non-Cooperative Environments based on Dynamic Time Camera-LiDAR Fusion

A real-time dynamic obstacle tracking and mapping system for UAV navigation and collision avoidance with an RGB-D camera

Autonomous quadrotor obstacle avoidance based on dueling double deep recurrent Q-learning with monocular vision

Real-time Obstacle Avoidance with Deep Reinforcement Learning Three-Dimensional Autonomous Obstacle Avoidance for UAV

Research on Surface Environment Perception Via Camera-LiDAR Sensor Fusion