Abstract:Numerous deep learning techniques have been explored in pursuit of achieving precise road segmentation; nonetheless, this task continues to present a significant challenge. Exposing shadows and the obstruction of objects are the most important difficulties associated with road segmentation using optical image data alone. By incorporating additional data sources, such as LiDAR data, the accuracy of road segmentation can be improved in areas where optical images are insufficient to segment roads properly. The missing information in spectral data due to the object blockage and shadow effect can be compensated by the integration of 2D and 3D information. This study proposes a feature-wise fusion strategy of optical images and point clouds to enhance the road segmentation performance of a deep learning model. For this purpose, high-resolution satellite images and airborne LiDAR point cloud collected over Florida, USA, were used. Eigenvalue-based and geometric 3D property-based features were calculated based on the LiDAR data. These optical images and LiDAR-based features were used together to train, end-to-end, a deep residual U-Net architecture. In this strategy, the high-level features generated from optical images were concatenated with the LiDAR-based features before the final convolution layer. The consistency of the proposed strategy was evaluated using ResNet backbones with a different number of layers. According to the obtained results, the proposed fusion strategy improved the prediction capacity of the U-Net models with different ResNet backbones. Regardless of the backbone, all models showed enhancement in prediction statistics by 1% to 5%. The combination of optical images and LiDAR point cloud in the deep learning model has increased the prediction performance and provided the integrity of road geometry in woodland and shadowed areas.

RGB Camera and LiDAR Fusion for Road Detection

RGB-LiDAR fusion for accurate 2D and 3D object detection

Multi-Stage Residual Fusion Network for LIDAR-Camera Road Detection

Road Segmentation with Image-LiDAR Data Fusion

Real time object detection using LiDAR and camera fusion for autonomous driving

Capitalizing on RGB-FIR Hybrid Imaging for Road Detection

AI Powered Road Network Prediction with Multi-Modal Data

RangeLVDet: Boosting 3D Object Detection in LIDAR With Range Image and RGB Image

Real-Time Fusion Network for RGB-D Semantic Segmentation Incorporating Unexpected Obstacle Detection for Road-Driving Images

Real-Time Vehicle Detection Framework Based on the Fusion of LiDAR and Camera

Deep LiDAR-Radar-Visual Fusion for Object Detection in Urban Environments

Fusion of neural networks, for LIDAR-based evidential road mapping

Enhanced Automotive Object Detection via RGB-D Fusion in a DiffusionDet Framework

Late sensor fusion approach with a designed multi-segmentation network

RI-Fusion: 3D Object Detection Using Enhanced Point Features With Range-Image Fusion for Autonomous Driving.

Deep Representation Learning for Road Detection Using Siamese Network.

ResNet-based Surface Normal Estimator with Multilevel Fusion Approach with Adaptive Median Filter Region Growth Algorithm for Road Scene Segmentation

Influence of Camera-LiDAR Configuration on 3D Object Detection for Autonomous Driving

Fusion of color images and LiDAR data for lane classification.

Improving Road Segmentation by Combining Satellite Images and LiDAR Data with a Feature-Wise Fusion Strategy

Real-Time Hybrid Multi-Sensor Fusion Framework for Perception in Autonomous Vehicles