Video-Based 3D pose estimation for residential roofing

Ruochen Wang,Liying Zheng,Ashley L. Hawke,Robert E. Carey,Scott P. Breloff,Kang Li,Xi Peng
DOI: https://doi.org/10.1080/21681163.2022.2072394
2022-05-19
Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization
Abstract:Residential roofers are often exposed to awkward postures and motions in a prolonged time, which may not only reduce their body stability and increase fall potential, but also increase the risk of musculoskeletal disorders (MSDs). To assess their risks of fatal and musculoskeletal injuries, it is crucial to capture 3D body poses of workers during roofing tasks. In this paper, we proposed a novel two-stage motion estimation approach based on a convolution neural network to estimate residential roofer's body poses using three-view video data. Our approach includes two stages: (1) use of an offline multi-view model to estimate the 3D pose in a single frame; (2) use of a multi-frame model to apply temporal convolutions to refine the multi-view outputs. The performance of the approach was evaluated by comparing our estimation with the gold-standard marker-based 3D human pose during one of the common residential roofing tasks – shingle installation. The evaluation results show that the proposed multi-frame model can effectively improve the accuracy of the coordinate sequence. Moreover, these results prove that the proposed video-based motion estimation approach can efficiently and accurately locate 3D body joints and pave the way for future onsite motion analysis during roofing activities.
What problem does this paper attempt to address?