Abstract:The collaborative visual perception of multiple Unmanned Aerial Vehicles (UAVs) has increasingly become a research hotspot. Compared to a single UAV equipped with a short-baseline stereo camera, multi-UAV collaborative vision offers a wide and variable baseline, providing potential benefits in flexible and large-scale depth perception. In this paper, we propose the concept of a collaborative stereo camera, where the left and right cameras are mounted on two UAVs that share an overlapping FOV. Considering the dynamic flight of two UAVs in the real world, the FOV and relative pose of the left and right cameras are continuously changing. Compared to fixed-baseline stereo cameras, this aerial collaborative stereo system introduces two challenges, which are highly real-time requirements for dynamic cross-camera stereo feature association and relative pose estimation of left and right cameras. To address these challenges, we first propose a real-time dual-channel feature association algorithm with a guidance-prediction structure. Then, we propose a Relative Multi-State Constrained Kalman Filter (Rel-MSCKF) algorithm to estimate the relative pose by fusing co-visual features and UAVs' visual-inertial odometry (VIO). Extensive experiments are performed on the popular onboard computer NVIDIA NX. Results on the resource-constrained platform show that the real-time performance of the dual-channel feature association is significantly superior to traditional methods. The convergence of Rel-MSCKF is assessed under different initial baseline errors. In the end, we present a potential application of aerial collaborative stereo for remote mapping obstacles in urban scenarios. We hope this work can serve as a foundational study for more multi-UAV collaborative vision research. Online video: <a class="link-external link-https" href="https://youtu.be/avxMuOf5Qcw" rel="external noopener nofollow">this https URL</a>

On-Line Cooperative Observation Based on ESMF in Three-Dimensional Enviroments

Target Detection Formulti-Uavs Via Digital Pheromones and Navigation Algorithm in Unknownenvironments

Real-Time Onboard 3D State Estimation of an Unmanned Aerial Vehicle in Multi-Environments Using Multi-Sensor Data Fusion

Monocular-GPS Fusion 3D Object Detection for UAVs

Integrated Design of Cooperative Area Coverage and Target Tracking with Multi-UAV System

Simultaneous Search and Tracking of Non-cooperative Mobile Targets Using Multiple UAVs in Uneven Environments

Observation-Driven Multiple UAV Coordinated Standoff Target Tracking Based on Model Predictive Control

Multi UAV-enabled Distributed Sensing: Cooperation Orchestration and Detection Protocol

Multiobjective coordinated search algorithm for swarm of UAVs based on 3D-simplified virtual forced model

On the Information Evolution and Sensor Deployment for Multi-Spacecraft Cooperative Visual 3D Sensing of Orbital Targets

Towards Aerial Collaborative Stereo: Real-Time Cross-Camera Feature Association and Relative Pose Estimation for UAVs

Multi-UAV Cooperative Coverage Search for Various Regions Based on Differential Evolution Algorithm

End-to-End Vision-Based Cooperative Target Geo-Localization for Multiple Micro UAVs

On the Information Evolution and Sensor Deployment for Multispacecraft Cooperative Visual 3-D Sensing of Orbital Targets

Searching and Tracking an Unknown Number of Targets: A Learning-Based Method Enhanced with Maps Merging

Research on the Cooperative Target State Estimation and Tracking Optimization Method of Multi-UUV

Application of ellipsoidal approximation into target tracking for multi UAVs formation cooperative detection

An Efficient Framework for Probabilistic Search with Multi-UAVs

Distributed Multi-Target Search and Surveillance Mission Planning for Unmanned Aerial Vehicles in Uncertain Environments

Real-Time Optimal States Estimation with Inertial and Delayed Visual Measurements for Unmanned Aerial Vehicles

3D on and Off-Grid Dynamic Channel Tracking for Multiple UAVs and Satellite Communications