Abstract:Introduction Sensory deficits are common in neurological disorders. Proprioception is an example of a sensory system that could be impaired. In clinical populations, proprioception is preferably assessed as easily as possible. Techniques based on deep learning to precisely track body landmarks in simple video recordings are promising to analyze movement in clinical settings, outside the laboratory and without the need to attach markers. This study aims to validate deep learning based motion capture (DeepLabCut) with respect to laboratory-based 3D optoelectronic motion capture (gold standard) to assess proprioception by measuring the Joint Reproduction Error (JRE) during a knee joint position sense test [1]. Research question Can deep learning based motion capture (DeepLabCut) provide valid assessment of proprioception that is comparable to the gold standard (laboratory-based 3D optoelectronic motion capture)? Methods Proprioception was assessed with the eyes closed during 45 knee position sense tests of 11 typically developing children (6 girls, 5 boys, mean age 8.9 ± 1.0 years old, mean BMI 17.0 ± 2.6kg/m^2) by determining the difference between the criterion and reproduced angle (JRE, in degrees). The joint angles were determined with 2 setups; 1) 3D optoelectronic motion capture (Vicon, 10 cameras) using the ISB lower limb marker model [2] (26 markers), and 2) 2D deep learning based motion capture (DeepLabCut [3,4]) in sagittal video files using the pre-trained human model (MPII) with 55 manually labeled frames from videos of children, which were used only for network training and were excluded from the statistical analyses. Negative knee JREs correspond to a greater reproduced angle compared to the criterion angle. Linear mixed models were used to assess differences in JREs between DeepLabCut and Vicon. In addition, the mean knee JRE was calculated per subject and compared between Vicon and DeepLabCut using a paired t-test. Results There was no significant difference in the knee JRE measured with Vicon (-4.18°) and the knee JRE measured with DeepLabCut (-4.03°) (F=0.01, p = 0.91) (Fig. 1). The mean JRE over repeated trials per subject determined by using Vicon was not significantly different from the mean JRE determined by using DeepLabCut (t(10) = -0.11, p = 0.92). Discussion Measuring the knee JRE with DeepLabCut seemed valid with respect to the gold standard in typically developing children. Further research is needed and ongoing to assess the validity of DeepLabCut to measure the JRE in other joints than the knee (such as the ankle and the hip) and assess the reliability of measuring the JRE with DeepLabCut. Furthermore, we are working on validation of DeepLabCut to assess proprioception using the joint position sense tests in children with Cerebral Palsy. Deep learning based motion capture techniques could facilitate the implementation of proprioception assessment in clinical settings as there are no markers and no laboratory setting needed.

Validation of DeepLabCut as a tool for markerless 3D pose estimation

DeepLabCut: markerless pose estimation of user-defined body parts with deep learning

Deep-Learning-Based Markerless Pose Estimation Systems in Gait Analysis: DeepLabCut Custom Training and the Refinement Function

Using DeepLabCut for tracking body landmarks in videos of children with dyskinetic cerebral palsy: a working methodology

MocapMe: DeepLabCut-Enhanced Neural Network for Enhanced Markerless Stability in Sit-to-Stand Motion Capture

On the inference speed and video-compression robustness of DeepLabCut

Marker-Less 3d Human Motion Capture With Monocular Image Sequence And Height-Maps

Anatomical-Marker-Driven 3D Markerless Human Motion Capture

Multi-animal pose estimation and tracking with DeepLabCut

Validity of deep learning based motion capture using DeepLabCut to assess proprioception

The accuracy of markerless motion capture combined with computer vision techniques for measuring running kinematics

The effect of the number of labelled frames on the accuracy of 2D markerless pose estimation (DeepLabCut) during treadmill walking

Multi-animal pose estimation, identification and tracking with DeepLabCut

Comparison of a Deep Learning-Based Pose Estimation System to Marker-Based and Kinect Systems in Exergaming for Balance Training

A Review of Deep Learning Techniques for Markerless Human Motion on Synthetic Datasets

A Deep Learning Model for Markerless Pose Estimation Based on Keypoint Augmentation: What Factors Influence Errors in Biomechanical Applications?

The Development and Evaluation of a Fully Automated Markerless Motion Capture Workflow

Validation & Exploration of Multimodal Deep-Learning Camera-Lidar Calibration models

A Deep Neural Network-based Method for Estimation of 3D Lifting Motions.

Concurrent validity and test reliability of the deep learning markerless motion capture system during the overhead squat

Beyond the Edge: Markerless Pose Estimation of Speech Articulators from Ultrasound and Camera Images Using DeepLabCut