Abstract:Bridge inspection is an important step in preserving and rehabilitating transportation infrastructure for extending their service lives. The advancement of mobile robotic technology allows the rapid collection of a large amount of inspection video data. However, the data are mainly images of complex scenes, wherein a bridge of various structural elements mix with a cluttered background. Assisting bridge inspectors in extracting structural elements of bridges from the big complex video data, and sorting them out by classes, will prepare inspectors for the element-wise inspection to determine the condition of bridges. This paper is motivated to develop an assistive intelligence model for segmenting multiclass bridge elements from inspection videos captured by an aerial inspection platform. With a small initial training dataset labeled by inspectors, a Mask Region-based Convolutional Neural Network (Mask R-CNN) pre-trained on a large public dataset was transferred to the new task of multiclass bridge element segmentation. Besides, the temporal coherence analysis attempts to recover false negatives and identify the weakness that the neural network can learn to improve. Furthermore, a semi-supervised self-training (S$^3$T) method was developed to engage experienced inspectors in refining the network iteratively. Quantitative and qualitative results from evaluating the developed deep neural network demonstrate that the proposed method can utilize a small amount of time and guidance from experienced inspectors (3.58 hours for labeling 66 images) to build the network of excellent performance (91.8% precision, 93.6% recall, and 92.7% f1-score). Importantly, the paper illustrates an approach to leveraging the domain knowledge and experiences of bridge professionals into computational intelligence models to efficiently adapt the models to varied bridges in the National Bridge Inventory.

A Deep Neural Network for Multiclass Bridge Element Parsing in Inspection Image Analysis

Computer Vision-Based Railway Viaduct Component Inspection Using Multi-task Multi-scale Deep Neural Network

Component Identification and Depth Estimation for Structural Images Based on Multi-Scale Task Interaction Network

Ultrasonic Array Tomography-Oriented Subsurface Crack Recognition and Cross-Section Image Reconstruction of Reinforced Concrete Structure Using Deep Neural Networks

Deep Learning‐based Automatic Classification of Three‐level Surface Information in Bridge Inspection

A Multitask Deep Learning Model for Parsing Bridge Elements and Segmenting Defect in Bridge Inspection Images

A semi-supervised self-training method to develop assistive intelligence for segmenting multiclass bridge elements from inspection videos

Post‐earthquake Damage Recognition and Condition Assessment of Bridges Using UAV Integrated with Deep Learning Approach

Automated crack detection and mapping of bridge decks using deep learning and drones

Multilevel Structural Components Detection and Segmentation toward Computer Vision-Based Bridge Inspection

A New Deep Learning Network for Automatic Bridge Detection from SAR Images Based on Balanced and Attention Mechanism

A Robot System for Rapid and Intelligent Bridge Damage Inspection Based on Deep-Learning Algorithms

Underwater Inspection of Bridge Substructures Using Sonar and Deep Convolutional Network

Autonomous Identification of Bridge Concrete Cracks Using Unmanned Aircraft Images and Improved Lightweight Deep Convolutional Networks

Vision-based Automated Bridge Component Recognition Integrated With High-level Scene Understanding

Region of interest (ROI) extraction and crack detection for UAV-based bridge inspection using point cloud segmentation and 3D-to-2D projection

Bridge Damage Detection using a Single-Stage Detector and Field Inspection Images

Automated bridge component recognition from point clouds using deep learning

Hierarchical Convolutional Neural Network with Feature Preservation and Autotuned Thresholding for Crack Detection

Image‐based post‐disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization

Deep inspection: an electrical distribution pole parts study via deep neural networks