Abstract:Estimating the relative pose between a camera and a LiDAR holds paramount importance in facilitating complex task execution within multi-agent systems. Nonetheless, current methodologies encounter two primary limitations. First, amid the cross-modal feature extraction, they typically employ separate modal branches to extract cross-modal features from images and point clouds. This approach results in the feature spaces of images and point clouds being misaligned, thereby reducing the robustness of establishing correspondences. Second, due to the scale differences between images and point clouds, one-to-many pixel-point correspondences are inevitably encountered, which will mislead the pose optimization. To address these challenges, we propose a framework named \(\text{I2P}_{\text{ppsim}}\) ( I mage to P oint cloud registration by learning the underlying alignment feature space from P ixel-to- P oint SIM ilarities). Central to \(\text{I2P}_{\text{ppsim}}\) is a shared feature alignment module (SFAM). It is designed under on a coarse-to-fine architecture and uses a weight-sharing network to construct an alignment feature space. Benefiting from SFAM, \(\text{I2P}_{\text{ppsim}}\) can effectively identify the co-view regions between images and point clouds and establish high-reliability 2D-3D correspondences. Moreover, to mitigate the one-to-many correspondence issue, we introduce a similarity maximization strategy termed point-max. This strategy effectively filters out outliers, thereby establishing accurate 2D-3D correspondences. To evaluate the efficacy of our framework, we conduct extensive experiments on KITTI Odometry and Oxford Robotcar. The results corroborate the effectiveness of our framework in improving image-to-point cloud registration. To make our results reproducible, the source codes have been released at https://cslinzhang.github.io/I2P .

I2D-Loc: Camera Localization Via Image to LiDAR Depth Flow

3D LiDAR-Based Global Localization Using Siamese Neural Network

I2D-Loc++: Camera Pose Tracking in LiDAR Maps with Multi-View Motion Flows

LocNet: Global Localization in 3D Point Clouds for Mobile Vehicles

End-to-end 2D-3D Registration between Image and LiDAR Point Cloud for Vehicle Localization

Deep Camera Pose Regression Using Pseudo-LiDAR

2D-3D Cross-Modality Network for End-to-End Localization with Probabilistic Supervision

DP-Loc: Visual Localization in 2D Maps Using an Embedded Depth Prior

LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignment

I2P Registration by Learning the Underlying Alignment Feature Space from Pixel-to-Point Similarities

BEVLOC: End-to-End 6-Dof Localization Via Cross-Modality Correlation under Bird’s Eye View

DeepI2P: Image-to-Point Cloud Registration via Deep Classification

PointLoc: Deep Pose Regressor for LiDAR Point Cloud Localization

(LC)$^2$: LiDAR-Camera Loop Constraints For Cross-Modal Place Recognition

LiDAR Odometry by Deep Learning-Based Feature Points with Two-Step Pose Estimation

NIDALoc: Neurobiologically Inspired Deep LiDAR Localization

EP2P-Loc: End-to-End 3D Point to 2D Pixel Localization for Large-Scale Visual Localization

Attention-Enhanced Cross-modal Localization Between Spherical Images and Point Clouds

LiDAR-Aided Visual-Inertial Localization with Semantic Maps

Relative geometry-aware siamese neural network for 6DOF camera relocalization

LodoNet: A Deep Neural Network with 2D Keypoint Matchingfor 3D LiDAR Odometry Estimation