Abstract:Accurate and fast extraction of step parameters from video recordings of gait allows for richer information to be obtained from clinical tests such as Timed Up and Go. Current deep-learning methods are promising, but lack in accuracy for many clinical use cases. Extracting step parameters will often depend on extracted landmarks (keypoints) on the feet. We hypothesize that such keypoints can be determined with an accuracy relevant for clinical practice from video recordings by combining an existing general-purpose pose estimation method (OpenPose) with custom convolutional neural networks (convnets) specifically trained to identify keypoints on the heel. The combined method finds keypoints on the posterior and lateral aspects of the heel of the foot in side-view and frontal-view images from which step length and step width can be determined for calibrated cameras. Six different candidate convnets were evaluated, combining three different standard architectures as networks for feature extraction (backbone), and with two different networks for predicting keypoints on the heel (head networks). Using transfer learning, the backbone networks were pre-trained on the ImageNet dataset, and the combined networks (backbone + head) were fine-tuned on data from 184 trials of older, unimpaired adults. The data was recorded at three different locations and consisted of 193 k side-view images and 110 k frontal-view images. We evaluated the six different models using the absolute distance on the floor between predicted keypoints and manually labelled keypoints. For the best-performing convnet, the median error was 0.55 cm and the 75% quartile was below 1.26 cm using data from the side-view camera. The predictions are overall accurate, but show some outliers. The results indicate potential for future clinical use by automating a key step in marker-less gait parameter extraction.

Obtaining lower-body Euler angle time series in an accurate way using depth camera relying on Optimized Kinect CNN

Estimating Lower Extremity Running Gait Kinematics with a Single Accelerometer: A Deep Learning Approach

Depth-aware pose estimation using deep learning for exoskeleton gait analysis

Kinect as a Tool for Gait Analysis: Validation of a Real-Time Joint Extraction Algorithm Working in Side View

3-D Markerless Tracking of Human Gait by Geometric Trilateration of Multiple Kinects

A Kinect-Based Motion Capture Method for Assessment of Lower Extremity Exoskeleton

Leveraging Two Kinect Sensors for Accurate Full-Body Motion Capture

Estimating Lower Limb Kinematics Using a Reduced Wearable Sensor Count

Evaluation of lower extremity gait analysis using Kinect V2® tracking system

Dual-channel cascade pose estimation network trained on infrared thermal image and groundtruth annotation for real-time gait measurement

Evaluation of the Pose Tracking Performance of the Azure Kinect and Kinect v2 for Gait Analysis in Comparison with a Gold Standard: A Pilot Study

Personalization of Wearable Sensor-Based Joint Kinematic Estimation Using Computer Vision for Hip Exoskeleton Applications

Tracking Human-like Natural Motion Using Deep Recurrent Neural Networks

Convolutional Neural Network for Estimating Spatiotemporal and Kinematic Gait Parameters using a Single Inertial Sensor

Design of an accurate wireless data logger for vibration analysis with Android interface.

Continuous Prediction of Lower-Limb Kinematics From Multi-Modal Biomedical Signals

Leveraging depth cameras and wearable pressure sensors for full-body kinematics and dynamics capture

Improving IMU-Based Prediction of Lower Limb Kinematics in Natural Environments Using Egocentric Optical Flow

Real-Time Human Motion Capture Based on Wearable Inertial Sensor Networks

Cascaded 3D Full-body Pose Regression from Single Depth Image at 100 FPS

Two-step deep-learning identification of heel keypoints from video-recorded gait