Abstract:Unmanned Aerial Vehicles (UAVs) exhibit superior maneuverability and large-scale movement capability in 3D space, making them ideal platforms for various autonomous applications. However, UAVs with cameras face challenges in 3D situational awareness, which limits their applications in 3D space, particularly in Lake Search and Rescue (LSAR) scenarios. In this paper, we propose a new LSAR Aerial 3D detection dataset (LA3D) and a pioneering end-to-end aerial Monocular-GPS Fusion 3D detector (MGF3D) to improve UAVs’ 3D understanding capability and mitigate the gap between research and real-world application. LA3D is collected by a well-designed hybrid-wing UAV platform on lake surface scenes, which has high-quality annotated 3D bounding boxes, aligned GPS altitudes, and comprehensive metrics. Moreover, MGF3D aggregates multi-modal features with rich geometric and semantic information and feeds them into the 3D detection head for final predictions. To achieve this, we design the Altitude Encoder (AE) module to extract geometric GPS altitude features and introduce Image Encoder (IE) to encode semantic image features respectively. The plug-and-play Monocular-GPS Fusion (MGF) module is proposed to fuse GPS altitude features with image features, which adaptively learns 3D space information and enhances long-distance depth estimation capability. The extensive experiments on the LA3D dataset validate that MGF3D significantly improves aerial 3D detection accuracy. Furthermore, the MGF3D is deployed on a UAV embedded platform to perform the challenging real-world LSAR flight task, demonstrating the effectiveness and practicality of our LA3D and MGF3D. We plan to release the dataset and code in here.

Autonomous Landing of a Helicopter UAV with a Ground-Based Multisensory Fusion System

Multi-sensor Data Fusion for UAV Landing Based on Federal Variational Bayesian Filtering

An Integrated UWB-IMU-Vision Framework for Autonomous Approaching and Landing of UAVs

A Ground-Based Optical System for Autonomous Landing of a Fixed Wing UAV

An Approach to Ground Target Localization for Uavs Based on Multi-Sensor Fusion

A Vision-Based Autonomous Landing Guidance Strategy for a Micro-UAV by the Modified Camera View

Multi-sensor Data Fusion for UAV Landing Guidance Based on Bayes Estimation

Height Estimation Method for Landing of Small Unmanned Helicopter Based on Monocular Vision

Ground-based Visual Guidance in Autonomous UAV Landing

Multi-sensor Data Fusion of UAV Landing System

Vision-Aided Semi-Autonomous Landing System for UAV

Vision navigation for unmanned helicopter autonomous landing

Stereo Vision Guiding For The Autonomous Landing Of Fixed-Wing Uavs: A Saliency-Inspired Approach

Fusion of Millimeter Wave Radar and Monocular Vision for Obstacle Detection Based on Rotor-Wing UAV

Monocular-GPS Fusion 3D Object Detection for UAVs

A UAV Autonomous Landing System Integrating Locating, Tracking, and Landing in the Wild Environment

Vision System for an Unmanned Helicopter Landing in Complex Environment

Robust Autonomous Landing of UAV in Non-Cooperative Environments based on Dynamic Time Camera-LiDAR Fusion

Robust Autonomous Landing of UAVs in Non-Cooperative Environments Based on Comprehensive Terrain Understanding

Multi-resolution Visual Positioning and Navigation Technique for Unmanned Aerial System Landing Assistance

Autonomous Landing Of A Quadrotor On An Ugv