Abstract:Estimating the relative pose between a camera and a LiDAR holds paramount importance in facilitating complex task execution within multi-agent systems. Nonetheless, current methodologies encounter two primary limitations. First, amid the cross-modal feature extraction, they typically employ separate modal branches to extract cross-modal features from images and point clouds. This approach results in the feature spaces of images and point clouds being misaligned, thereby reducing the robustness of establishing correspondences. Second, due to the scale differences between images and point clouds, one-to-many pixel-point correspondences are inevitably encountered, which will mislead the pose optimization. To address these challenges, we propose a framework named \(\text{I2P}_{\text{ppsim}}\) ( I mage to P oint cloud registration by learning the underlying alignment feature space from P ixel-to- P oint SIM ilarities). Central to \(\text{I2P}_{\text{ppsim}}\) is a shared feature alignment module (SFAM). It is designed under on a coarse-to-fine architecture and uses a weight-sharing network to construct an alignment feature space. Benefiting from SFAM, \(\text{I2P}_{\text{ppsim}}\) can effectively identify the co-view regions between images and point clouds and establish high-reliability 2D-3D correspondences. Moreover, to mitigate the one-to-many correspondence issue, we introduce a similarity maximization strategy termed point-max. This strategy effectively filters out outliers, thereby establishing accurate 2D-3D correspondences. To evaluate the efficacy of our framework, we conduct extensive experiments on KITTI Odometry and Oxford Robotcar. The results corroborate the effectiveness of our framework in improving image-to-point cloud registration. To make our results reproducible, the source codes have been released at https://cslinzhang.github.io/I2P .

I2P Registration by Learning the Underlying Alignment Feature Space from Pixel-to-Point Similarities

Em-Based Point to Plane Icp for 3d Simultaneous Localization and Mapping

Probabilistic 3D ICP algorithm based on ORB feature

CoFiI2P: Coarse-to-Fine Correspondences for Image-to-Point Cloud Registration

Map Alignment Based on Plicp Algorithm for Multi-Robot Slam

DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registration

End-to-end 2D-3D Registration between Image and LiDAR Point Cloud for Vehicle Localization

An Automatic Registration Approach to Laser Point Sets Based on Multidiscriminant Parameter Extraction

OL-Reg: Registration of Image and Sparse LiDAR Point Cloud with Object-Level Dense Correspondences

CFI2P: Coarse-to-Fine Cross-Modal Correspondence Learning for Image-to-Point Cloud Registration

LiPO: LiDAR Inertial Odometry for ICP Comparison

I2P-Rec: Recognizing Images on Large-scale Point Cloud Maps through Bird's Eye View Projections

KISS-ICP: In Defense of Point-to-Point ICP -- Simple, Accurate, and Robust Registration If Done the Right Way

IGE-LIO: Intensity Gradient Enhanced Tightly Coupled LiDAR-Inertial Odometry

Pixels and 3-D Points Alignment Method for the Fusion of Camera and LiDAR Data.

One Framework to Register Them All: PointNet Encoding for Point Cloud Alignment

MaFreeI2P: A Matching-Free Image-to-Point Cloud Registration Paradigm with Active Camera Pose Retrieval

MF-LIO: Integrating Multi-Feature LiDAR Inertial Odometry with FPFH Loop Closure in SLAM

TFCT-I2P: Three stream fusion network with color aware transformer for image-to-point cloud registration

SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations

A Pose Association Method for Multi-Agent System Based on LiDAR Pointcloud Registration Algorithm