Abstract:Developing approaches for recognition and classification of awkward working postures is of great significance for proactive management of safety risks and work-related musculoskeletal disorders (WMSDs) in construction. Previous efforts have concentrated on wearable sensors or computer vision-based monitoring. However, certain limitations need to be further investigated. First, wearable sensor-based studies lack reliability due to vulnerability to environmental interferences. Second, conventional computer vision-based recognition demonstrates classification inaccuracy under adverse environmental conditions, such as insufficient illumination and occlusion. To address the above limitations, this study presents an innovative and automated approach for recognizing and classifying awkward working postures. This approach leverages multimodal data collected from various sensors and apparatuses, allowing for a comprehensive analysis of different modalities. A feature-level fusion strategy is employed to train deep learning-based networks, including a multilayer perceptron (MLP), recurrent neural network (RNN), and long short-term memory (LSTM). Among these networks, the LSTM model achieves optimal performance, with an impressive accuracy of 99.6% and an F1-score of 99.7%. A comparison of metrics between single-modality and multimodal-fused training methods demonstrates that the incorporation of multimodal fusion significantly enhances the classification performance. Furthermore, the study examines the performance of the LSTM network under adverse environmental conditions. The accuracy of the model remains consistently above 90% in such conditions, indicating that the model's generalizability is enhanced through the multimodal fusion strategy. In conclusion, this study mainly contributes to the body of knowledge on proactive prevention for safety and health risks in the construction industry by offering an automated approach with excellent adaptability in adverse conditions. Moreover, this innovative attempt integrating diverse data through multimodal fusion may provide inspiration for future studies to achieve advancements. Mitigating potential risk factors for work-related musculoskeletal disorders (WMSD) in construction and improving safety and health performance are crucial in construction projects. Construction workers are frequently exposed to prolonged periods of awkward working postures. In pursuit of a more comprehensive solution, a pioneering and automated approach for the recognition and classification of such postures is developed. Specifically, this approach is rooted in the use of joint point data extracted from RGB images, synergistically fused with motion data representing the activity state and electroencephalogram (EEG) data representing the cognitive state. Rigorously tested, this approach demonstrates remarkable classification performance in deep-learning networks, boasting a maximum accuracy of 99.6%. Such high accuracy substantiates its potential for implementation in real construction management. Considering the inherent complexities of dynamic construction sites, compounded by challenging environmental conditions such as insufficient illumination and occlusion, automated identification methods commonly confront limitations in utility. In response, the integrated approach in this study amalgamates the rich information derived from diverse modalities, ensuring a sustained high accuracy rate of 94.9%. This not only demonstrates the exceptional performance of the new approach, but also its generalizability, thereby enabling proactive management of ergonomic and safety risks in construction sites.

Deep learning-based construction equipment operators' mental fatigue classification using wearable EEG sensor data

Identification and Classification of Construction Equipment Operators' Mental Fatigue Using Wearable Eye-Tracking Technology

Identifying mental fatigue of construction workers using EEG and deep learning

Non-invasive detection of mental fatigue in construction equipment operators through geometric measurements of facial features

Validity of facial features’ geometric measurements for real-time assessment of mental fatigue in construction equipment operators

Internet of things and ensemble learning-based mental and physical fatigue monitoring for smart construction sites

Pre-service Fatigue Screening for Construction Workers Through Wearable EEG-based Signal Spectral Analysis

Automated and Continuous Fatigue Monitoring in Construction Workers Using Forearm EMG and IMU Wearable Sensors and Recurrent Neural Network

Classifying Tasks Performed by Electrical Line Workers Using a Wrist-Worn Sensor: A Data Analytic Approach

Monitoring workers' attention and vigilance in construction activities through a wireless and wearable electroencephalography system

Pre-Service Fatigue Screening for Construction Labor Through Hybrid Kinematic-EEG Signal Processing and Workload Assessments

Deep learning-based networks for automated recognition and classification of awkward working postures in construction using wearable insole sensor data

Physical Exertion Modeling for Construction Tasks Using Combined Cardiorespiratory and Thermoregulatory Measures

A Feature-Level Fusion-Based Multimodal Analysis of Recognition and Classification of Awkward Working Postures in Construction

Identification of mental fatigue in language comprehension tasks based on EEG and deep learning

Recognizing sitting activities of excavator operators using multi-sensor data fusion with machine learning and deep learning algorithms

Real-Time Monitoring of Mental Fatigue of Construction Workers Using Enhanced Sequential Learning and Timeliness

Electroencephalography Based Fatigue Detection Using a Novel Feature Fusion and Extreme Learning Machine

Field Experiment on Wearable EEG/PPG-based Monitoring of Mental Fatigue in Drillers: Impact of Prolonged Working Hours and Night Shifts

Miner Fatigue Detection from Electroencephalogram-Based Relative Power Spectral Topography Using Convolutional Neural Network

Towards Mental Load Assessment for High-Risk Works Driven by Psychophysiological Data: Combining a 1D-CNN Model with Random Forest Feature Selection