Abstract:Multi-modal systems enhance performance in autonomous driving but face inefficiencies due to indiscriminate processing within each modality. Additionally, the independent feature learning of each modality lacks interaction, which results in extracted features that do not possess the complementary characteristics. These issue increases the cost of fusing redundant information across modalities. To address these challenges, we propose targeting driving-relevant elements, which reduces the volume of LiDAR features while preserving critical information. This approach enhances lane level interaction between the image and LiDAR branches, allowing for the extraction and fusion of their respective advantageous features. Building upon the camera-only framework PHP, we introduce the Lane-level camera-LiDAR Fusion Planning (LFP) method, which balances efficiency with performance by using lanes as the unit for sensor fusion. Specifically, we design three modules to enhance efficiency and performance. For efficiency, we propose an image-guided coarse lane prior generation module that forecasts the region of interest (ROI) for lanes and assigns a confidence score, guiding LiDAR processing. The LiDAR feature extraction modules leverages lane-aware priors from the image branch to guide sampling for pillar, retaining essential pillars. For performance, the lane-level cross-modal query integration and feature enhancement module uses confidence score from ROI to combine low-confidence image queries with LiDAR queries, extracting complementary depth features. These features enhance the low-confidence image features, compensating for the lack of depth. Experiments on the Carla benchmarks show that our method achieves state-of-the-art performance in both driving score and infraction score, with maximum improvement of 15% and 14% over existing algorithms, respectively, maintaining high frame rate of 19.27 FPS.

Lane-level Localization System Using Surround-View Cameras Adaptive to Different Driving Conditions

Persistent Stereo Visual Localization on Cross-Modal Invariant Map

Multimodal Localization: Stereo over LiDAR Map

Semantic Segmentation-Based Lane-Level Localization Using Around View Monitoring System

A lane-level localization method via the lateral displacement estimation model on expressway

L3: Sensing Driving Conditions for Vehicle Lane-Level Localization on Highways.

Lane-Level Vehicular Localization Utilizing Smartphones

Vehicles Lane-changing Behavior Detection

Lane Detection and Lane Departure Warning System

LFP: Efficient and Accurate End-to-End Lane-Level Planning via Camera-LiDAR Fusion

LaIF: A Lane-Level Self-Positioning Scheme for Vehicles in GNSS-Denied Environments

Real-Time Global Localization of Robotic Cars in Lane Level Via Lane Marking Detection and Shape Registration.

Robust Multilane Detection and Tracking in Urban Scenarios Based on LIDAR and Mono-Vision

3D Lane Detection from Front or Surround-View using Joint-Modeling & Matching

A Low Cost Sensors Approach for Accurate Vehicle Localization and Autonomous Driving Application

Lane Departure Assessment via Enhanced Single Lane-Marking

Vision-Based Lane-Changing Behavior Detection Using Deep Residual Neural Network

Robust Lane-Detection Method for Low-Speed Environments

Camera-Based Lane-Aided Multi-Information Integration for Land Vehicle Navigation

A Lane Change Detection and Filtering Approach for Precise Longitudinal Position of On-Road Vehicles

Global Navigation Satellite System/Inertial Measurement Unit/Camera/HD Map Integrated Localization for Autonomous Vehicles in Challenging Urban Tunnel Scenarios