Railway LiDAR semantic segmentation based on intelligent semi-automated data annotation

Florian Wulff,Bernd Schaeufele,Julian Pfeifer,Ilja Radusch

2024-10-17

Abstract:Automated vehicles rely on an accurate and robust perception of the environment. Similarly to automated cars, highly automated trains require an environmental perception. Although there is a lot of research based on either camera or LiDAR sensors in the automotive domain, very few contributions for this task exist yet for automated trains. Additionally, no public dataset or described approach for a 3D LiDAR semantic segmentation in the railway environment exists yet. Thus, we propose an approach for a point-wise 3D semantic segmentation based on the 2DPass network architecture using scans and images jointly. In addition, we present a semi-automated intelligent data annotation approach, which we use to efficiently and accurately label the required dataset recorded on a railway track in Germany. To improve performance despite a still small number of labeled scans, we apply an active learning approach to intelligently select scans for the training dataset. Our contributions are threefold: We annotate rail data including camera and LiDAR data from the railway environment, transfer label the raw LiDAR point clouds using an image segmentation network, and train a state-of-the-art 3D LiDAR semantic segmentation network efficiently leveraging active learning. The trained network achieves good segmentation results with a mean IoU of 71.48% of 9 classes.

Computer Vision and Pattern Recognition,Systems and Control

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the LiDAR semantic segmentation problem in the railway environment. Specifically, the author aims to develop a 3D LiDAR semantic segmentation method based on intelligent semi - automatic data annotation to support the highly automated train's perception ability in complex environments. The following are the main problems and solutions proposed in the paper:

### Main Problems
1. **Lack of Public Datasets**: Currently, there are no publicly available 3D LiDAR semantic segmentation datasets in the railway environment, which limits the progress of research.
2. **Environmental Perception Requirements**: Similar to self - driving cars, highly automated trains also need to accurately and robustly perceive the surrounding environment, especially when facing safety - critical tasks such as obstacle detection and emergency braking systems.
3. **Low Annotation Efficiency**: Manual annotation of large - scale LiDAR point - cloud data is time - consuming and error - prone, so efficient annotation tools and methods are required.

### Solutions
1. **Data Pre - processing and Synchronization**:
- Filter, denoise, and remove reflection points from the original LiDAR data, and synchronize it with camera images to ensure that the time - stamp difference is less than 10 milliseconds.
- Correct motion distortion to ensure the true proportion and flatness of static structures.
2. **Intelligent Semi - automatic Data Annotation**:
- Use the trained DeepLabV3+ network to perform semantic segmentation on camera images and map 2D labels to the nearest 3D LiDAR points.
- Manually correct mis - annotated labels due to inaccurate calibration and create a small high - quality dataset containing 52 finely - annotated scans.
- Use the FLLT.AI tool to visualize and annotate 3D point clouds, and to map and convert between 2D and 3D labels.
3. **Model Training**:
- Use the 2DPass network for the 3D LiDAR semantic segmentation task. 2DPass combines 2D and 3D network paths and uses 2D image data as prior information to enhance the semantic segmentation effect of 3D point clouds.
- For the initial training, use 602 roughly - annotated scan data and start from the pre - trained weights of the SemanticKITTI dataset through transfer learning.
- Introduce an Active Learning strategy, select the most uncertain and diverse scan data for re - annotation and training, and gradually improve the model performance.
4. **Evaluation Results**:
- After multiple iterative trainings, the final model achieved an average intersection - over - union (mIoU) of 71.48% and a frequency - weighted intersection - over - union (fwIoU) of 81.20% on the test set, which is significantly better than the initial training results.
- In particular, there has been a significant improvement in the classification of small objects (such as telegraph poles, traffic lights, and pedestrians), which is crucial for safety applications in the railway environment.

Through these methods, the author has successfully demonstrated how to achieve high - precision 3D LiDAR semantic segmentation in the railway environment under limited data conditions, and has provided new research directions and technical foundations for future automated train scene understanding.