Abstract:In recent years, computer vision has transformed fields such as medical imaging, object recognition, and geospatial analytics. One of the fundamental tasks in computer vision is semantic image segmentation, which is vital for precise object delineation. Autonomous driving represents one of the key areas where computer vision algorithms are applied. The task of road surface segmentation is crucial in self-driving systems, but it requires a labor-intensive annotation process in several data domains. The work described in this paper aims to improve the efficiency of image segmentation using a convolutional neural network in a multi-sensor setup. This approach leverages lidar (Light Detection and Ranging) annotations to directly train image segmentation models on RGB images. Lidar supplements the images by emitting laser pulses and measuring reflections to provide depth information. However, lidar's sparse point clouds often create difficulties for accurate object segmentation. Segmentation of point clouds requires time-consuming preliminary data preparation and a large amount of computational resources. The key innovation of our approach is the masked loss, addressing sparse ground-truth masks from point clouds. By calculating loss exclusively where lidar points exist, the model learns road segmentation on images by using lidar points as ground truth. This approach allows for blending of different ground-truth data types during model training. Experimental validation of the approach on benchmark datasets shows comparable performance to a high-quality image segmentation model. Incorporating lidar reduces the load on annotations and enables training of image-segmentation models without loss of segmentation quality. The methodology is tested on diverse datasets, both publicly available and proprietary. The strengths and weaknesses of the proposed method are also discussed in the paper.

Using layer-wise training for Road Semantic Segmentation in Autonomous Cars

Unifying Terrain Awareness Through Real-Time Semantic Segmentation

Semantic Segmentation and Depth Estimation of Urban Road Scene Images Using Multi-Task Networks

Implementation of a Lightweight Semantic Segmentation Algorithm in Road Obstacle Detection

Small obstacles detection on roads scenes using semantic segmentation for the safe navigation of autonomous vehicles

Semantic Segmentation with Transfer Learning for Off-Road Autonomous Driving

Lidar Annotation Is All You Need

Road Segmentation in SAR Satellite Images with Deep Fully-Convolutional Neural Networks

Efficient lightweight residual network for real-time road semantic segmentation

A novel approach for semantic segmentation of automatic road network extractions from remote sensing images by modified UNet

Research on road semantic segmentation based on hybrid auto-encoder

Reinforced Wasserstein Training for Severity-Aware Semantic Segmentation in Autonomous Driving

End to End Video Segmentation for Driving : Lane Detection For Autonomous Car

In Defense of Pre-trained ImageNet Architectures for Real-time Semantic Segmentation of Road-driving Images

An Automatic Road Surface Segmentation in Non-Urban Environments: A 3D Point Cloud Approach With Grid Structure and Shallow Neural Networks

PP-LinkNet: Improving Semantic Segmentation of High Resolution Satellite Imagery with Multi-stage Training

Learning Nighttime Semantic Segmentation the Hard Way

RoadSegNet: a deep learning framework for autonomous urban road detection

ENet-21: An Optimized light CNN Structure for Lane Detection

Learning from Maps: Visual Common Sense for Autonomous Driving

A Deep Learning-Based Semantic Segmentation Architecture for Autonomous Driving Applications