Abstract:Sensing and reality capture devices are widely used in construction sites. Among different technologies, vision-based sensors are by far the most common and ubiquitous. A large volume of images and videos is collected from construction projects every day to track work progress, measure productivity, litigate claims, and monitor safety compliance. Manual interpretation of such colossal amounts of data, however, is non-trivial, error-prone, and resource-intensive. This has motivated new research on soft computing methods that utilize high-power data processing, computer vision, and deep learning (DL) in the form of convolutional neural networks (CNNs). A fundamental step toward machine-driven interpretation of construction site scenery is to accurately identify objects of interest for a particular problem. The accuracy requirement, however, may offset the computational speed of the candidate method. While lightweight DL algorithms (e.g., Mask R-CNN) can perform visual recognition with relatively high accuracy, they suffer from low processing efficacy, which hinders their use in real-time decision-making. One of the most promising DL algorithms that balance speed and accuracy is YOLO (you-only-look-once). This paper investigates YOLO-based CNN models in fast detection of construction objects. First, a large-scale image dataset, named Pictor-v2, is created, which contains about 3,500 images and approximately 11,500 instances of common construction site objects (e.g., building, equipment, worker). To assess the agility of object detection, transfer learning is used to train two variations of this model, namely, YOLO-v2 and YOLO-v3, and test them on different data combinations (crowdsourced, web-mined, or both). Results indicate that performance is higher if the model is trained on both crowdsourced and web-mined images. Additionally, YOLO-v3 outperforms YOLO-v2 by focusing on smaller, harder-to-detect objects. The best-performing YOLO-v3 model has a 78.2% mAP when tested on crowdsourced data. Sensitivity analysis of the output shows that the model's strong suit is in detecting larger objects in less crowded and well-lit spaces. The proposed methodology can also be extended to predict the relative distance of the detected objects with reliable accuracy. Findings of this work lay the foundation for further research on technology-assistive systems to augment human capacities in quickly and reliably interpreting visual data in complex environments.

Content-Based Image Retrieval for Construction Site Images: Leveraging Deep Learning–Based Object Detection

Semi-supervised learning approach for construction object detection by integrating super-resolution and mean teacher network

Vision-based method for semantic information extraction in construction by integrating deep learning object detection and image captioning

Applying Deep Learning and Single Shot Detection in Construction Site Image Recognition

Construction site image retrieval based on material cluster recognition

Deep semantic segmentation for visual understanding on construction sites

Recognizing Diverse Construction Activities in Site Images Via Relevance Networks of Construction-Related Objects Detected by Convolutional Neural Networks

A Semi-Supervised Learning Detection Method for Vision-Based Monitoring of Construction Sites by Integrating Teacher-Student Networks and Data Augmentation

Deep Convolutional Networks for Construction Object Detection Under Different Visual Conditions

Deep Learning for Content-Based Image Retrieval: A Comprehensive Study

Image-based Onsite Object Recognition for Automatic Crane Lifting Tasks

Construction Activity Recognition Method Based on Object Detection, Attention Orientation Estimation, and Person Re-Identification

A Novel Approach for Annotation-based Image Retrieval Using Deep Architecture.

Real time detection method and device for different types of entity objects in construction site image

Few-shot object detection and attribute recognition from construction site images for improved field compliance

Deep Learning Detection for Real-time Construction Machine Checking

Development of an Image Data Set of Construction Machines for Deep Learning Object Detection

Vision-based Method for Tracking Workers by Integrating Deep Learning Instance Segmentation in Off-Site Construction

A Deep Learning Fusion Approach to Retrieve Images of People's Unsafe Behavior from Construction Sites

Deep-learning-based visual data analytics for smart construction management

A vision-based approach for detecting occluded objects in construction sites