Abstract:The collaborative visual perception of multiple Unmanned Aerial Vehicles (UAVs) has increasingly become a research hotspot. Compared to a single UAV equipped with a short-baseline stereo camera, multi-UAV collaborative vision offers a wide and variable baseline, providing potential benefits in flexible and large-scale depth perception. In this paper, we propose the concept of a collaborative stereo camera, where the left and right cameras are mounted on two UAVs that share an overlapping FOV. Considering the dynamic flight of two UAVs in the real world, the FOV and relative pose of the left and right cameras are continuously changing. Compared to fixed-baseline stereo cameras, this aerial collaborative stereo system introduces two challenges, which are highly real-time requirements for dynamic cross-camera stereo feature association and relative pose estimation of left and right cameras. To address these challenges, we first propose a real-time dual-channel feature association algorithm with a guidance-prediction structure. Then, we propose a Relative Multi-State Constrained Kalman Filter (Rel-MSCKF) algorithm to estimate the relative pose by fusing co-visual features and UAVs' visual-inertial odometry (VIO). Extensive experiments are performed on the popular onboard computer NVIDIA NX. Results on the resource-constrained platform show that the real-time performance of the dual-channel feature association is significantly superior to traditional methods. The convergence of Rel-MSCKF is assessed under different initial baseline errors. In the end, we present a potential application of aerial collaborative stereo for remote mapping obstacles in urban scenarios. We hope this work can serve as a foundational study for more multi-UAV collaborative vision research. Online video: <a class="link-external link-https" href="https://youtu.be/avxMuOf5Qcw" rel="external noopener nofollow">this https URL</a>

Research on Stereo Image Synthesis Based on Oblique Photography of UAV

A new stereo matching algorithm for UAV autonomous navigation

Spectral Analysis of Aerial Light Field for Optimization Sampling and Rendering of Unmanned Aerial Vehicle

An adaptive 3D reconstruction method for asymmetric dual-angle multispectral stereo imaging system on UAV platform

Towards Real-time 3D Reconstruction using Consumer UAVs

Parallel-Perspective Stereo Mosaics Of Video Images From Unmanned Aerial Vehicle Remote Sensing System

Stereoscopic Video Synthesis from A Monocular Video

Towards Aerial Collaborative Stereo: Real-Time Cross-Camera Feature Association and Relative Pose Estimation for UAVs

A novel approach of efficient 3D reconstruction for real scene using unmanned aerial vehicle oblique photogrammetry with five cameras

3D Feature Points Reconstruction Based on Stereo Vision of UAV

3d Stereoscopic Image Generation From Monocular View In Structured Environments

UAV image stitching by estimating orthograph with RGB cameras

Real-Time 3D Video Synthesis from Binocular Stereo Camera

Aerial orthoimage generation for UAV remote sensing: Review

Building 3D Realistic Modeling Based on Air-ground Multi-source Data Fusion

High Resolution 3-D Imaging Via Distributed UAVs SAR Tomography

AN OBJECT-ORIENTED UAV 3D PATH PLANNING METHOD APPLIED IN CULTURAL HERITAGE DOCUMENTATION

APPLICATION OF UAV OBLIQUE PHOTOGRAPHY IN REAL SCENE 3D MODELING

An Aerial Composite Imaging Method with Multiple Upright Cameras Based on Axis-Shift Theory