Abstract:Operator attention failure due to mental fatigue during extended equipment operations is a common cause of equipment-related accidents that result in catastrophic injuries and fatalities. As a result, tracking operators' mental fatigue is critical to reducing equipment-related accidents on construction sites. Previously, several strategies aimed at recognizing mental fatigue with adequate accuracy, such as machine learning utilizing EEG-based wearable sensing systems, have been proposed. However, the ability to track operators' mental fatigue for its implementation on an actual construction site is still an issue. For instance, the mobility and systemic instability of EEG sensors necessitate their application in laboratory settings rather than on actual construction sites. Furthermore, while the machine learning classifiers achieved acceptable accuracy, their input is limited to manually developed EEG features, which may compromise the models' performance on real construction sites. Accordingly, the current research proposes the viability of a construction site strategy that uses flexible headband-based sensors for acquiring raw EEG data and deep learning networks to recognize operators' mental fatigue. To serve this purpose, a one-hour excavator operation by fifteen operators was conducted on a construction site. The NASA-TLX score was used as the ground truth of mental fatigue, and brain activity patterns were recorded using a wearable EEG sensor. The raw EEG data was then used to develop deep learning-based classification models. Finally, the performance of deep learning models, i.e., long short-term memory, bidirectional LSTM, and one-dimensional convolutional networks, was investigated using accuracy, precision, recall, specificity, and an F1-score. The findings indicate that the Bi-LSTM model outperforms the other deep learning models with a high accuracy of 99.941% and F1-score between 99.917% and 99.993%. These findings demonstrate the feasibility of applying the Bi-LSTM model and contribute to wearable sensor-based mental fatigue recognition and classification, thus enhancing on-site health and safety operations.

Automating excavator productivity measurement using deep learning

A New Measurement Method of Real-time Pose Estimation for an Automatic Hydraulic Excavator

A deep learning based solution for construction equipment detection: from development to deployment

Automatic vision-based calculation of excavator earthmoving productivity using zero-shot learning activity recognition

Deep learning-based recognition of construction activities in real construction site environment

Intelligent Technologies for Construction Machinery Using Data-Driven Methods

Automatic Identification of Idling Reasons in Excavation Operations Based on Excavator–Truck Relationships

LSTM-Based Workload Recognition for Hydraulic Actuators: A Case Study on Excavator Digging Process

Automatic Mass Estimation of Construction Vehicles by Modeling Operational and Engine Data

Automated Estimation of Construction Equipment Emission Using Inertial Sensors and Machine Learning Models

Automated Activity Recognition of Construction Equipment Using a Data Fusion Approach

Deep learning-based construction equipment operators' mental fatigue classification using wearable EEG sensor data

Material Recognition for Automated Progress Monitoring using Deep Learning Methods

Recognizing sitting activities of excavator operators using multi-sensor data fusion with machine learning and deep learning algorithms

Deep learning technology for construction machinery and robotics

Construction site safety monitoring and excavator activity analysis system

Multi-camera vision-based productivity monitoring of earthmoving operations

An imitation from observation approach for dozing distance learning in autonomous bulldozer operation

Data integration using deep learning and real-time locating system (RTLS) for automated construction progress monitoring and reporting

A Software-Based Approach for Acoustical Modeling of Construction Job Sites with Multiple Operational Machines

Critical Review and Road Map of Automated Methods for Earthmoving Equipment Productivity Monitoring