Abstract:Multi-modal systems enhance performance in autonomous driving but face inefficiencies due to indiscriminate processing within each modality. Additionally, the independent feature learning of each modality lacks interaction, which results in extracted features that do not possess the complementary characteristics. These issue increases the cost of fusing redundant information across modalities. To address these challenges, we propose targeting driving-relevant elements, which reduces the volume of LiDAR features while preserving critical information. This approach enhances lane level interaction between the image and LiDAR branches, allowing for the extraction and fusion of their respective advantageous features. Building upon the camera-only framework PHP, we introduce the Lane-level camera-LiDAR Fusion Planning (LFP) method, which balances efficiency with performance by using lanes as the unit for sensor fusion. Specifically, we design three modules to enhance efficiency and performance. For efficiency, we propose an image-guided coarse lane prior generation module that forecasts the region of interest (ROI) for lanes and assigns a confidence score, guiding LiDAR processing. The LiDAR feature extraction modules leverages lane-aware priors from the image branch to guide sampling for pillar, retaining essential pillars. For performance, the lane-level cross-modal query integration and feature enhancement module uses confidence score from ROI to combine low-confidence image queries with LiDAR queries, extracting complementary depth features. These features enhance the low-confidence image features, compensating for the lack of depth. Experiments on the Carla benchmarks show that our method achieves state-of-the-art performance in both driving score and infraction score, with maximum improvement of 15% and 14% over existing algorithms, respectively, maintaining high frame rate of 19.27 FPS.

Fast and Robust Approaches for Lane Detection Using Multi‐camera Fusion in Complex Scenes

Multiple Lane Detection Via Combining Complementary Structural Constraints

LFP: Efficient and Accurate End-to-End Lane-Level Planning via Camera-LiDAR Fusion

A Novel Lane Line Detection Based on Multi-feature Fusion and Windows Searching

Multi-Modal Attention Guided Real-Time Lane Detection

A Method of Lane Departure Identification Based on Roadside Multi-Sensor Fusion

Real-time Structured Lane Detection by Monocular Vision

Deep Multi-Sensor Lane Detection

LHFFNet: A hybrid feature fusion method for lane detection

Efficient Multi-Lane Detection Based on Large-Kernel Convolution and Location

DV-3DLane: End-to-end Multi-modal 3D Lane Detection with Dual-view Representation

Multimodal Fusion Network for 3-D Lane Detection

Robust Lane-Detection Method for Low-Speed Environments

Fast Multi-Lane Detection Based on CNN Differentiation for ADAS/AD

Combining Low-Light Scene Enhancement for Fast and Accurate Lane Detection

Fusion of color images and LiDAR data for lane classification.

Robust Lane Detection from Continuous Driving Scenes Using Deep Neural Networks

Multi-Stage Residual Fusion Network for LIDAR-Camera Road Detection

Real-Time Vehicle Detection Framework Based on the Fusion of LiDAR and Camera

A Fast and Robust Lane Detection via Online Re-Parameterization and Hybrid Attention

A Hybrid Spatial-temporal Deep Learning Architecture for Lane Detection