Abstract:The detection and prevention of workers’ body straining postures and other stressing conditions within the work environment, supports establishing occupational safety and promoting well being and sustainability at work. Developed methods towards this aim typically rely on combining highly ergonomic workplaces and expensive monitoring mechanisms including wearable devices. In this work, we demonstrate how the input from low-cost sensors, specifically, passive camera sensors installed in a real manufacturing workplace, and smartwatches used by the workers can provide useful feedback on the workers’ conditions and can yield key indicators for the prevention of work-related musculo-skeletal disorders (WMSD) and physical fatigue. To this end, we study the ability to assess the risk for physical strain of workers online during work activities based on the classification of ergonomically sub-optimal working postures using visual information, the correlation and fusion of these estimations with synchronous worker heart rate data, as well as the prediction of near-future heart rate using deep learning-based techniques. Moreover, a new multi-modal dataset of video and heart rate data captured in a real manufacturing workplace during car door assembly activities is introduced. The experimental results show the efficiency of the proposed approach that exceeds 70% of classification rate based on the F1 score measure using a set of over 300 annotated video clips of real line workers during work activities. In addition a time lagging correlation between the estimated ergonomic risks for physical strain and high heart rate was assessed using a larger dataset of synchronous visual and heart rate data sequences. The statistical analysis revealed that imposing increased strain to body parts will results in an increase to the heart rate after 100–120 s. This finding is used to improve the short term forecasting of worker’s cardiovascular activity for the next 10 to 30 s by fusing the heart rate data with the estimated ergonomic risks for physical strain and ultimately to train better predictive models for worker fatigue.

Machine Learning Techniques for Motion Analysis of Fatigue from Manual Material Handling Operations Using 3D Motion Capture Data

A Data-Driven Fatigue Prediction using Recurrent Neural Networks

Machine Learning Assisted Human Fatigue Detection, Monitoring, and Recovery: Review

A Forecasting Framework for Predicting Perceived Fatigue: Using Time Series Methods to Forecast Ratings of Perceived Exertion with Features from Wearable Sensors

Wearable network for multilevel physical fatigue prediction in manufacturing workers

Automated Workers Ergonomic Risk Assessment in Manual Material Handling using sEMG Wearable Sensors and Machine Learning

Closed-Chain Inverse Dynamics for the Biomechanical Analysis of Manual Material Handling Tasks through a Deep Learning Assisted Wearable Sensor Network

Automated Workers’ Ergonomic Risk Assessment in Manual Material Handling Using sEMG Wearable Sensors and Machine Learning

Using Non-invasive Wearable Sensors to Estimate Perceived Fatigue Level in Manual Material Handling Tasks

A new muscle fatigue and recovery model and its ergonomics application in human simulation

Framework for Dynamic Evaluation of Muscle Fatigue in Manual Handling Work

Deep Learning for Fatigue Estimation on the Basis of Multimodal Human-Machine Interactions

Human Reliability Modeling in Occupational Environments Toward a Safe and Productive Operator 4.0

Assessing Fatigue with Multimodal Wearable Sensors and Machine Learning

Proving the Potential of Skeleton Based Action Recognition to Automate the Analysis of Manual Processes

An Automatic and Non-Invasive Physical Fatigue Assessment Method for Construction Workers

Latent Space Representation of Human Movement: Assessing the Effects of Fatigue

Detection of Physical Strain and Fatigue in Industrial Environments Using Visual and Non-Visual Low-Cost Sensors

An Evaluation Method of Assessing the Low Back Muscle Fatigue in Manual Material Handling

A framework for evaluating muscle activity during repetitive manual material handling in construction manufacturing

Analysis of Manual Materials Handling Fatigue Based on sEMG and ECG