Abstract:Recent advances in mapping techniques have enabled the creation of highly accurate dense 3D maps during robotic missions, such as point clouds, meshes, or NeRF-based representations. These developments present new opportunities for reusing these maps for localization. However, there remains a lack of a unified approach that can operate seamlessly across different map representations. This paper presents and evaluates a global visual localization system capable of localizing a single camera image across various 3D map representations built using both visual and lidar sensing. Our system generates a database by synthesizing novel views of the scene, creating RGB and depth image pairs. Leveraging the precise 3D geometric map, our method automatically defines rendering poses, reducing the number of database images while preserving retrieval performance. To bridge the domain gap between real query camera images and synthetic database images, our approach utilizes learning-based descriptors and feature detectors. We evaluate the system's performance through extensive real-world experiments conducted in both indoor and outdoor settings, assessing the effectiveness of each map representation and demonstrating its advantages over traditional structure-from-motion (SfM) localization approaches. The results show that all three map representations can achieve consistent localization success rates of 55% and higher across various environments. NeRF synthesized images show superior performance, localizing query images at an average success rate of 72%. Furthermore, we demonstrate an advantage over SfM-based approaches that our synthesized database enables localization in the reverse travel direction which is unseen during the mapping process. Our system, operating in real-time on a mobile laptop equipped with a GPU, achieves a processing rate of 1Hz.

TrafficLoc: Localizing Traffic Surveillance Cameras in 3D Scenes

LocNet: Global Localization in 3D Point Clouds for Mobile Vehicles

3D LiDAR-Based Global Localization Using Siamese Neural Network

CenterLoc3D: Monocular 3D Vehicle Localization Network for Roadside Surveillance Cameras

Global Localization Based on Road-centric 3D Point Cloud Descriptor in Urban Environments

CAROM -- Vehicle Localization and Traffic Scene Reconstruction from Monocular Cameras on Road Infrastructures

Monocular Visual Object 3D Localization in Road Scenes

Crowdsourced 3D Mapping: A Combined Multi-View Geometry and Self-Supervised Learning Approach

Traffic Intersection Re-Identification Using Monocular Camera Sensors

Vehicle 3d Localization in Road Scenes VIA a Monocular Moving Camera

Sparse Semantic Map-Based Monocular Localization in Traffic Scenes Using Learned 2D-3D Point-Line Correspondences

Attention-Enhanced Cross-modal Localization Between Spherical Images and Point Clouds

Vehicle Localization in 3D World Coordinates Using Single Camera at Traffic Intersection

CrossLoc: Scalable Aerial Localization Assisted by Multimodal Synthetic Data

Multiple-Kernel Based Vehicle Tracking Using 3D Deformable Model and Camera Self-Calibration

MapLocNet: Coarse-to-Fine Feature Registration for Visual Re-Localization in Navigation Maps

Visual Localization in 3D Maps: Comparing Point Cloud, Mesh, and NeRF Representations

Influence of Camera-LiDAR Configuration on 3D Object Detection for Autonomous Driving

Collaborative Localization Based on Traffic Landmarks for Autonomous Driving

Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image

PLReg3D: Learning 3D Local and Global Descriptors Jointly for Global Localization