Deep learning-based activity-aware 3D human motion trajectory prediction in construction
Moein Younesi Heravi,Youjin Jang,Inbae Jeong,Sajib Sarkar
DOI: https://doi.org/10.1016/j.eswa.2023.122423
IF: 8.5
2023-11-11
Expert Systems with Applications
Abstract:Predicting human motion is a critical requirement in various applications, with particular significance in the construction sector. This task presents significant challenges due to the diverse nature of human actions and the complexities of converting 2D image coordinates to real-world space. In response to these challenges, this paper introduces an innovative deep learning-driven approach to forecast human motion trajectories, with a novel emphasis on activity recognition to improve predictive accuracy. The method utilizes a multi-camera system to extract 2D joint locations, which are then fused using a particle filter technique for 3D pose generation. Using 3D data, deep learning models are developed to first recognize the activity class, and then take it as auxiliary information for predicting the motion trajectory. Through a comprehensive experiment, we evaluated the proposed methodology. While the main innovation of the proposed approach lies in the incorporation of deep learning-based activity recognition into the trajectory prediction system, the experiment results revealed the activity-aware system's capability to enhance prediction performance by a minimum of 6.4% and up to 16.6% in short-term forecasts, in compare with a conventional approach. Additionally, we analyzed the effects of varying time windows and joint selections on predictive outcomes across diverse scenarios and discussed the implications of these findings. By enhancing the prediction of human motion, this approach holds promise in improving workspace safety while encouraging effective interactions within complex environments.
computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science