Abstract:Maintaining the roadway infrastructure is one of the essential factors in enabling a safe, economic, and sustainable transportation system. Manual roadway damage data collection is laborious and unsafe for humans to perform. This area is poised to benefit from the rapid advance and diffusion of artificial intelligence technologies. Specifically, deep learning advancements enable the detection of road damages automatically from the collected road images. This work proposes to collect and label road damage data using Google Street View and use YOLOv7 (You Only Look Once version 7) together with coordinate attention and related accuracy fine-tuning techniques such as label smoothing and ensemble method to train deep learning models for automatic road damage detection and classification. The proposed approaches are applied to the Crowdsensing-based Road Damage Detection Challenge (CRDDC2022), IEEE BigData 2022. The results show that the data collection from Google Street View is efficient, and the proposed deep learning approach results in F1 scores of 81.7% on the road damage data collected from the United States using Google Street View and 74.1% on all test images of this dataset.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the automation of road damage detection and classification. Specifically, traditional methods of collecting road damage data are both labor - intensive and unsafe. Therefore, the author proposes a method to automatically detect and classify road damage by using Google Street View images and YOLOv7 (You Only Look Once version 7) and its related techniques (such as coordinate attention mechanism and label smoothing, etc.). This method aims to improve the efficiency of data collection, reduce costs, increase frequency and improve accuracy, so as to better respond to road problems, reduce maintenance and other economic costs, and enhance the safety of citizens. ### Main research objectives: 1. **Efficient collection and labeling of road damage data**: Use Google Street View images to collect and label road damage data to build a data set for training deep - learning models. 2. **Explore advanced object detection methods**: Use the current state - of - the - art object detection methods (such as YOLOv7) and related techniques (such as coordinate attention mechanism and label smoothing) to improve the performance of road damage detection and classification. 3. **Experimental verification**: Conduct experiments on the Crowdsensing - based Road Damage Detection Challenge (CRDDC2022) data set to evaluate the performance of the proposed model. ### Specific problems solved: - **Inefficiency and insecurity of manual data collection**: Traditional methods of manually collecting road damage data are time - consuming and have potential safety hazards. - **Limitations of existing models**: Existing deep - learning models have poor generalization ability between different countries and regions and need to be optimized for road types in specific regions. - **Difficulties in data labeling**: Manually labeling a large amount of road damage data is very time - consuming and error - prone. This burden can be alleviated by using existing data sets and Google Street View images. ### Technical means: - **Data collection**: Download images from Google Street View to build a data set containing road damage in the United States. - **Model selection**: Use YOLOv7 as the base model and introduce the coordinate attention mechanism and label smoothing techniques to improve the detection and classification performance of the model. - **Data augmentation**: By adjusting the image augmentation parameters, generate more realistic and diverse training samples to improve the robustness of the model. - **Model optimization**: Use the integration method and test - time augmentation techniques to further improve the prediction accuracy of the model. ### Experimental results: - **F1 score**: The experimental results on the CRDDC2022 data set show that the F1 score of the proposed model on the US data set reaches 81.7%, and the average F1 score on the overall data set is 66.3%. - **Ranking**: This method won the third place in the Crowdsensing - based Road Damage Detection Challenge (CRDDC2022). ### Conclusion: By using Google Street View images and advanced deep - learning techniques, this research has successfully improved the automation level of road damage detection and classification, providing a more efficient, accurate and safe solution for road management. Future work will further expand the data set, try more model configurations, and optimize the inference time of the model.

Road Damages Detection and Classification with YOLOv7

Optimizing YOLO Architectures for Optimal Road Damage Detection and Classification: A Comparative Study from YOLOv7 to YOLOv10

Road disease detection algorithm based on YOLOv5s-DSG

Embedded System for Road Damage Detection by Deep Convolutional Neural Network

A Computer Vision-assisted Approach to Automated Real-Time Road Infrastructure Management

Automated Road Damage Detection Using UAV Images and Deep Learning Techniques

Road Damage Detection using Deep Ensemble Learning

Street View Image-Based Road Marking Inspection System Using Computer Vision and Deep Learning Techniques

Improved YOLOv5-Based Real-Time Road Pavement Damage Detection in Road Infrastructure Management

Road Damage Detection and Classification with Detectron2 and Faster R-CNN

YOLOv5s-M: A deep learning network model for road pavement damage detection from urban street-view imagery

Road damage detection method based on improved YOLOv8n

Cloud-Based Collaborative Road-Damage Monitoring with Deep Learning and Smartphones

FasterRCNN Monitoring of Road Damages: Competition and Deployment

Deep learning-based road damage detection and classification for multiple countries

Road Damage Detection and Classification Using Deep Neural Networks with Smartphone Images

YOLOv8-LSD: Improved YOLOv8 Focused on Small Target Information Extraction for Road Damage Detection

YOLOv8-PD: an improved road damage detection algorithm based on YOLOv8n model

Deep Learning-based Road Crack Detection Technology

Road Surface Defect Detection Algorithm Based on YOLOv8

Research on road Damage Detection Based on Improved YOLOv8