Abstract:Robot localization using a previously built map is essential for a variety of tasks including highly accurate navigation and mobile manipulation. A popular approach to robot localization is based on image-to-point cloud registration, which combines illumination-invariant LiDAR-based mapping with economical image-based localization. However, the recent works for image-to-point cloud registration either divide the registration into separate modules or project the point cloud to the depth image to register the RGB and depth images. In this paper, we present I2PNet, a novel end-to-end 2D-3D registration network. I2PNet directly registers the raw 3D point cloud with the 2D RGB image using differential modules with a unique target. The 2D-3D cost volume module for differential 2D-3D association is proposed to bridge feature extraction and pose regression. 2D-3D cost volume module implicitly constructs the soft point-to-pixel correspondence on the intrinsic-independent normalized plane of the pinhole camera model. Moreover, we introduce an outlier mask prediction module to filter the outliers in the 2D-3D association before pose regression. Furthermore, we propose the coarse-to-fine 2D-3D registration architecture to increase localization accuracy. We conduct extensive localization experiments on the KITTI Odometry and nuScenes datasets. The results demonstrate that I2PNet outperforms the state-of-the-art by a large margin. In addition, I2PNet has a higher efficiency than the previous works and can perform the localization in real-time. Moreover, we extend the application of I2PNet to the camera-LiDAR online calibration and demonstrate that I2PNet outperforms recent approaches on the online calibration task.

LocNet: Global Localization in 3D Point Clouds for Mobile Vehicles

3D LiDAR-Based Global Localization Using Siamese Neural Network

Geo-Localization with Transformer-Based 2D-3D Match Network

LiDAR-Based Global Localization Using Histogram of Orientations of Principal Normals

Multimodal Localization: Stereo over LiDAR Map

Persistent Stereo Visual Localization on Cross-Modal Invariant Map

Communication Constrained Cloud-Based Long-Term Visual Localization in Real Time.

One RING to Rule Them All: Radon Sinogram for Place Recognition, Orientation and Translation Estimation

Global Localization Based on Road-centric 3D Point Cloud Descriptor in Urban Environments

Global Localization with a Single-Line LiDAR by Dense 2D Signature and 1D Registration

Global Localization in Large-scale Point Clouds via Roll-pitch-yaw Invariant Place Recognition and Low-overlap Global Registration

PLReg3D: Learning 3D Local and Global Descriptors Jointly for Global Localization

TrafficLoc: Localizing Traffic Surveillance Cameras in 3D Scenes

PointLoc: Deep Pose Regressor for LiDAR Point Cloud Localization

AGL-NET: Aerial-Ground Cross-Modal Global Localization with Varying Scales

End-to-end 2D-3D Registration between Image and LiDAR Point Cloud for Vehicle Localization

Pole-like Objects Mapping and Long-Term Robot Localization in Dynamic Urban Scenarios

2D-3D Cross-Modality Network for End-to-End Localization with Probabilistic Supervision

CapsLoc3D: Point Cloud Retrieval for Large-Scale Place Recognition Based on 3D Capsule Networks

InstaLoc: One-shot Global Lidar Localisation in Indoor Environments through Instance Learning

Autonomous Vehicle Localization with Prior Visual Point Cloud Map Constraints in GNSS-Challenged Environments