Abstract:In air-ground collaboration scenarios without GPS and prior maps, the relative positioning of drones and unmanned ground vehicles (UGVs) has always been a challenge. For a drone equipped with monocular camera and an UGV equipped with LiDAR as an external sensor, we propose a robust and real-time relative pose estimation method (LVCP) based on the tight coupling of vision and LiDAR point cloud information, which does not require prior information such as maps or precise initial poses. Given that large-scale point clouds generated by 3D sensors has more accurate spatial geometric information than the feature point cloud generated by image, we utilize LiDAR point clouds to correct the drift in visual-inertial odometry (VIO) when the camera undergoes significant shaking or the IMU has a low signal-to-noise ratio. To achieve this, we propose a novel coarse-to-fine framework for LiDAR-vision collaborative localization. In this framework, we construct point-plane association based on spatial geometric information, and innovatively construct a point-aided Bundle Adjustment (BA) problem as the backend to simultaneously estimate the relative pose of the camera and LiDAR and correct the VIO drift. In this process, we propose a particle swarm optimization (PSO) based sampling algorithm to complete the coarse estimation of the current camera-LiDAR pose. In this process, the initial pose of the camera used for sampling is obtained based on VIO propagation, and the valid feature-plane association number (VFPN) is used to trigger PSO-sampling process. Additionally, we propose a method that combines Structure from Motion (SFM) and multi-level sampling to initialize the algorithm, addressing the challenge of lacking initial values.

What problem does this paper attempt to address?

The paper primarily aims to address the problem of relative positioning between Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs) in the absence of GPS signals and prior maps. Specifically, the research proposes a real-time relative pose estimation method based on tightly coupled LiDAR and visual information (LVCP), aiming to achieve high-precision relative positioning between UAVs and UGVs. ### Core Issues Addressed by the Paper: 1. **Relative Positioning Challenge**: How to accurately determine the relative position between UAVs and UGVs in the absence of GPS signals and prior maps. 2. **Sensor Fusion**: How to effectively fuse monocular camera data from UAVs with LiDAR data from UGVs to improve positioning accuracy and robustness. 3. **Initialization Problem**: How to effectively initialize the system in the absence of precise initial relative pose. 4. **Drift Correction**: How to use LiDAR point cloud information to correct drift in Visual-Inertial Odometry (VIO), especially when the camera experiences severe shaking or the Inertial Measurement Unit (IMU) has a low signal-to-noise ratio. ### Main Contributions 1. **Coarse-to-Fine Framework**: A coarse-to-fine framework is proposed to achieve real-time LiDAR-visual collaborative positioning, including coarse positioning through Particle Swarm Optimization (PSO) and precise positioning by constructing a Point-aided Bundle Adjustment (BA) problem. 2. **Adaptive Sampling Algorithm**: An adaptive event-triggered PSO sampling algorithm is proposed to achieve relatively accurate point cloud matching within a limited time. 3. **Point-aided Optimization Structure**: Based on 3D geometric spatial information, feature point clouds are associated with LiDAR point clouds, and a point-aided BA problem is constructed based on point-plane residuals. 4. **Multi-level Sampling Method**: A multi-level sampling method is combined with Structure from Motion (SFM) feature points and LiDAR maps to address the lack of initial values. 5. **Experimental Validation**: Comparative experiments were conducted using open-source datasets and self-built datasets to verify the effectiveness, real-time performance, robustness, and accuracy of the algorithm, and it was extended to multi-UAV scenarios. In summary, the LVCP system proposed in this paper can achieve accurate relative positioning between UAVs and UGVs in unknown environments and effectively solve the scale estimation errors and trajectory drift problems caused by monocular visual information.

LVCP: LiDAR-Vision Tightly Coupled Collaborative Real-time Relative Positioning

LODM: Large-scale Online Dense Mapping for UAV

Ct-LVI: A Framework Toward Continuous-Time Laser-Visual-Inertial Odometry and Mapping

Robust Visual-Lidar Simultaneous Localization and Mapping System for UAV

RLP‐VIO: Robust and Lightweight Plane‐based Visual‐inertial Odometry for Augmented Reality

Visual Localization in a Prior 3D LiDAR Map Combining Points and Lines

LVIO-SAM: A Multi-sensor Fusion Odometry via Smoothing and Mapping

LiDAR-Visual-Inertial Odometry Based on Optimized Visual Point-Line Features

JVLDLoc: a Joint Optimization of Visual-LiDAR Constraints and Direction Priors for Localization in Driving Scenario.

JVLDLoc: a Joint Optimization of Visual-LiDAR Constraints and Direction Priors for Localization in Driving Scenario

Pose Estimation Based on Bidirectional Visual–Inertial Odometry with 3D LiDAR (BV-LIO)

Fusion of Visual-Inertial Odometry with LiDAR Relative Localization for Cooperative Guidance of a Micro-Scale Aerial Vehicle

Ct-LVI: A Framework Towards Continuous-time Laser-Visual-Inertial Odometry and Mapping

Vanishing Point Aided LiDAR-Visual-Inertial Estimator

A Collaborative Relative Localization Method for Vehicles using Vision and LiDAR Sensors

LVIO-Fusion:Tightly-Coupled LiDAR-Visual-Inertial Odometry and Mapping in Degenerate Environments

VE-LIOM: A Versatile and Efficient LiDAR-Inertial Odometry and Mapping System

Stereo and LiDAR Loosely Coupled SLAM Constrained Ground Detection

An Aerial and Ground Multi-Agent Cooperative Location Framework in GNSS-Challenged Environments

Cross-modal LiDAR-Visual-Inertial Localization in Pre-built LiDAR Point Cloud Map Through Direct Projection

Jointly Optimized Global-Local Visual Localization of UAVs