Abstract:Numerous navigation applications rely on data from global navigation satellite systems (GNSS), even though their accuracy is compromised in urban areas, posing a significant challenge, particularly for precise autonomous car localization. Extensive research has focused on enhancing localization accuracy by integrating various sensor types to address this issue. This paper introduces a novel approach for car localization, leveraging image features that correspond with highly detailed semantic 3D building models. The core concept involves augmenting positioning accuracy by incorporating prior geometric and semantic knowledge into calculations. The work assesses outcomes using Level of Detail 2 (LoD2) and Level of Detail 3 (LoD3) models, analyzing whether facade-enriched models yield superior accuracy. This comprehensive analysis encompasses diverse methods, including off-the-shelf feature matching and deep learning, facilitating thorough discussion. Our experiments corroborate that LoD3 enables detecting up to 69\% more features than using LoD2 models. We believe that this study will contribute to the research of enhancing positioning accuracy in GNSS-denied urban canyons. It also shows a practical application of under-explored LoD3 building models on map-based car positioning.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to explore the impact of semantic LoD3 building models on image - based vehicle localization. Specifically, it attempts to solve the problem of how to improve vehicle localization accuracy in urban environments, especially when GNSS signals are blocked. #### Background and Challenges 1. **Blocked GNSS Signals**: The Global Navigation Satellite System (GNSS) performs poorly in complex environments such as urban canyons, resulting in a decline in vehicle localization accuracy. 2. **Multi - sensor Fusion**: To solve this problem, much research has focused on improving localization accuracy by combining different types of sensors. 3. **Limitations of Existing Methods**: Although previous studies have utilized LoD1 and LoD2 building models for vehicle localization, these models lack detailed facade information, which limits their application in complex environments. #### Research Objectives The paper introduces a new method to enhance image - based vehicle localization accuracy by using highly detailed semantic 3D building models, especially LoD3 models. The main objectives include: - **Evaluating the Effect of LoD3 Models**: Analyzing whether LoD3 models provide more feature points than LoD2 models, thereby improving localization accuracy. - **Combining Visual Information with 3D Models**: Combining the visual information in optical images with the non - textured surface structures of semantic 3D models to achieve more accurate localization. - **Experimental Verification**: Comparing the performance of building models with different levels of detail (LoD2 and LoD3) in vehicle localization through experiments to verify the advantages of LoD3 models. #### Main Contributions 1. **Vehicle Localization Supported by High - Fidelity LoD3 Models**: Using high - fidelity LoD3 models and images for map - assisted vehicle localization. 2. **Cross - Modal Fusion**: Coordinating two different modalities: the visual information in optical images and the non - textured surface structures of semantic 3D models. 3. **Model Comparison**: Comparing the advantages and disadvantages of LoD3 and LoD2 building models in camera localization. Through these studies, the author hopes to provide new ideas for improving localization accuracy in the case of blocked GNSS signals in urban canyons and demonstrate the practical application potential of LoD3 building models in map - based vehicle localization.

Analyzing the impact of semantic LoD3 building models on image-based vehicle localization

3D LiDAR-Based Global Localization Using Siamese Neural Network

LESS-Map: Lightweight and Evolving Semantic Map in Parking Lots for Long-term Self-Localization

LocNet: Global Localization in 3D Point Clouds for Mobile Robots.

Semantic Image Alignment for Vehicle Localization

BDLoc: Global Localization from 2.5D Building Map

From Satellite to Ground: Satellite Assisted Visual Localization with Cross-view Semantic Matching

Efficient LiDAR Odometry for Autonomous Driving.

Visual Semantic Localization based on HD Map for Autonomous Vehicles in Urban Scenarios

Combining visibility analysis and deep learning for refinement of semantic 3D building models by conflict classification

Scan2LoD3: Reconstructing semantic 3D building models at LoD3 using ray casting and Bayesian networks

Monocular Vehicle Self-localization Method Based on Compact Semantic Map

Monocular Localization with Semantics Map for Autonomous Vehicles

Visual-Marker-Based Localization for Flat-Variation Scene

Semantic localization on BIM-generated maps using a 3D LiDAR sensor

Utilization of Semantic Planes: Improved Localization and Dense Semantic Map for Monocular SLAM in Urban Environment

Towards Localizing Structural Elements: Merging Geometrical Detection with Semantic Verification in RGB-D Data

A Survey on 3D LiDAR Localization for Autonomous Vehicles

LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignment

CenterLoc3D: Monocular 3D Vehicle Localization Network for Roadside Surveillance Cameras

Towards Urban Scene Semantic Segmentation with Deep Learning from LiDAR Point Clouds: A Case Study in Baden-Württemberg, Germany