Abstract:Vision-based anomaly inspection plays a crucial role in the efficient maintenance of millions of kilometers of railway, with rail segmentation, a key step in such anomaly detection for providing localization prior. However multi-type rails, those involved in crossings and connections, have highly variable patterns, greatly restricting the performance of standard (straight) rail segmentation methods. Semantic segmentation helps to deal with complex railway scenes and variable patterns, however the noise sensitivity, intra-class differences, and inter-class similarities still challenge the segmentation. Superpixel segmentation can aggregate local similar pixels with precise boundaries, which can offer a weak prior for semantic segmentation for boundary information modeling, intra-class aggregation, and inter-class differentiation, however how to integrate superpixel-level guidance to advance rail segmentation is still challenging. This paper proposes a two-stage transformer-Convolutional Neural Network (CNN)-based segmentation framework. The first stage, Attention-Based Superpixel Segmentation Sub-Network via Boundary Calibration (BCASN), generates railway superpixels by the learning of intra-superpixel consistency and boundary calibration to effectively fit rail boundaries and guide the second-stage rail segmentation. The second stage, Superpixel-Guided Multi-Type Rail Segmentation Sub-Network via Contextual Information Aggregation (CIASSN), captures railway semantics via global and cross-scale context construction, aggregates rail features via directional guidance and structured prior, and makes comprehensive segmentation decisions at superpixel and pixel scales with the learning of superpixel-level context and classification. The experiments demonstrate that the proposed solution achieves 98.71% overall accuracy, 98.44% mIoU, and 87.33% boundary recall in multi-type rail segmentation, significantly extends applicable scenarios, and outperforms all related state-of-the-art methods in rail and road segmentation.

Superpixel-Guided Multi-Type Rail Segmentation Via Contextual Information Aggregation

Computer Vision-Based Railway Viaduct Component Inspection Using Multi-task Multi-scale Deep Neural Network

RailSeg: Learning Local–Global Feature Aggregation With Contextual Information for Railway Point Cloud Semantic Segmentation

Hybrid deep learning architecture for rail surface segmentation and surface defect detection

Superpixel-Based Long-Range Dependent Network for High-Resolution Remote-Sensing Image Classification

Semantic Segmentation of Remote Sensing Images Depicting Environmental Hazards in High-Speed Rail Network Based on Large-Model Pre-Classification

Mcnet: Multiple Context Information Segmentation Network Of No-Service Rail Surface Defects

Rail Surface Defect Detection Using A Transformer-Based Network

An Efficient Algorithm for Extracting Railway Tracks Based on Spatial-Channel Graph Convolutional Network and Deep Neural Residual Network

Automatic Rail Surface Defects Inspection Based on Mask R-CNN

RBNet: An Ultra Fast Rendering-based Architecture for Railway Defects Segmentation

Efficient Dual-Stream Fusion Network for Real-Time Railway Scene Understanding

Classification and extraction method of hidden dangers along railway lines based on semantic segmentation network

Automatic railroad track components inspection using real‐time instance segmentation

Region and Edge-Aware Network for Rail Surface Defect Segmentation

Segmentation of Track Surface Defects Based on Machine Vision and Neural Networks

RailSegVITNet: A lightweight VIT-based real-time track surface segmentation network for improving railroad safety

An Object Segmentation Method for Railway Surrounding Environment Based on Three-dimensional Point Cloud

No-Service Rail Surface Defect Segmentation via Normalized Attention and Dual-scale Interaction

Edge-Enabled Real-time Railway Track Segmentation

A novel transformer-based semantic segmentation framework for structural condition assessment