Abstract:With the recent advancements in deep learning and computer vision, the AI-powered construction machine such as autonomous excavator has made significant progress. Safety is the most important section in modern construction, where construction machines are more and more automated. In this paper, we propose a vision-based excavator perception, activity analysis, and safety monitoring system. Our perception system could detect multi-class construction machines and humans in real-time while estimating the poses and actions of the excavator. Then, we present a novel safety monitoring and excavator activity analysis system based on the perception result. To evaluate the performance of our method, we collect a dataset using the Autonomous Excavator System (AES) (Zhang et al., Sci Robot 6(55):eabc3164) including multi-class of objects in different lighting conditions with human annotations. We also evaluate our method on a benchmark construction dataset. The results showed our YOLO v5 multi-class objects detection model improved inference speed by 8 times (YOLO v5 x-large) to 34 times (YOLO v5 small) compared with Faster R-CNN/YOLO v3 model (Zhang et al., In Proceedings of the 38th International Symposium on Automation and Robotics in Construction 461 (ISARC), pp. 49–56. InternationalAssociation for Automation and Robotics in Construction (IAARC), Dubai, UAE (2021). https://doi.org/10.22260/ISARC2021/0009). Furthermore, the accuracy of YOLO v5 models is improved by 2.7% (YOLO v5 x-large) while model size is reduced by 63.9% (YOLO v5 x-large) to 93.9% (YOLO v5 small). The experimental results show that the proposed action recognition approach outperforms the state-of-the-art approaches on top-1 accuracy by about 5.18%. The proposed real-time safety monitoring system is not only designed for our Autonomous Excavator System (AES) in solid waste scenes, it can also be applied to general construction scenarios.

Video-based construction vehicles detection and its application in intelligent monitoring system

Detection of engineering vehicles in high-resolution monitoring images

&Lt;title>automatic Traffic Real-Time Analysis System Based on Video</title>

A Video-Based Traffic Information Extraction System

Vehicles Detection and Tracking Based on the Sub-Feature

Hydraulic Excavators Recognition Based on Inverse "V" Feature of Mechanical Arm.

Intelligent Video Surveillance for Checking Attendance of Traffic Controllers in Level Crossing

Dump Truck Recognition Based On Scpsr In Videos

Detection of traffic congestion in road-occupied electric power construction based on video recognition

DESIGN SAFETY MONITORING SYSTEM FOR INTELLIGENT TUNNEL TRAFFIC BASED ON VIDEO DETECTION PROCESSING

Construction site safety monitoring and excavator activity analysis system

Video Monitoring System Application to Urban Traffic Intersection.

Robust Vehicle Extraction in Video-Based Intelligent Transportation Systems

Moving Vehicle Detection for Automatic Traffic Monitoring

Detection and Tracking of Vehicles and Event Analysis from Video

Design of the Safety Monitoring System for Civil Engineering Construction

Automatic Identification of Idling Reasons in Excavation Operations Based on Excavator–Truck Relationships

Application of Image Processing and Three-Dimensional Data Reconstruction Algorithm Based on Traffic Video in Vehicle Component Detection

Design on Video-Based Automatic Detecting and Recording System of Vehicles for Violation of Traffic Signal

Vehicle Detection and Tracking in Remote Sensing Satellite Vidio Based on Dynamic Association.

Real-time smart video surveillance to manage safety: A case study of a transport mega-project