Abstract:Background: Finger mobility plays a crucial role in everyday living and is a leading indicator during hand rehabilitation and assistance tasks. Depth-based hand pose estimation is a potentially low-cost solution for the clinical and home-based measurement of symptoms of limited human finger motion. Objective: The purpose of this study was to achieve the contactless measurement of finger motion based on depth-based hand pose estimation using Azure Kinect depth cameras and transfer learning, and to evaluate the accuracy in comparison with a three-dimensional motion analysis (3DMA) system. Methods: Thirty participants performed a series of tasks during which their hand motions were measured concurrently using the Azure Kinect and 3DMA systems. We propose a simple and effective approach to achieving real-time hand pose estimations from single depth images using ensemble convolutional neural networks trained by a transfer learning strategy. Algorithms to calculate the finger joint motion angles are presented by tracking the locations of the 24 hand joints. To demonstrate their potential, the Azure-Kinect-based 3D finger motion measurement system and algorithms are experimentally verified through comparison with a camera-based 3DMA system, which is the gold standard. Results: Our results revealed that the Azure-Kinect-based hand pose estimation system produced highly correlated measurements of hand joint coordinates. Our method achieved excellent performance in terms of the fraction of good frames ( > 80%) when the error thresholds were larger than approximately 2 cm, and the range of mean error distance was 0.23 - - 1.05 cm. For joint angles, the Azure Kinect and 3DMA systems had comparable inter-trial reliability (ICC2,1 ranging from 0.89 to 0.97) and excellent concurrent validity, with Pearsons r-values > 0.90 for most measurements (range: 0.88 - - 0.97). The 95% BlandAltman limits of agreement were narrow enough for the Azure Kinect to be considered a valid tool for the measurement of all reported joint angles of the index finger and thumb in pinching. Moreover, our method runs in real time at over 45 fps. Conclusion: The results of this study suggest that the proposed method has the capacity to measure the performance of fine motor skills.

Effect of Skin Movement on the Measurement Accuracy of Human Hand 3-D Motion

Hand-Finger Pose Estimation Using Inertial Sensors, Magnetic Sensors and a Magnet

Precision and Accuracy of Measuring Finger Motion with a Depth Camera: a Cross-Sectional Study of Healthy Participants

Estimate Hand–Finger Position with One Magnetometer and Known Relative Orientation

A 3-D Finger Motion Measurement System Via Soft Strain Sensors for Hand Rehabilitation

Principal Component Analysis of Motion Patterns for the Point-Touch Movement of the Index Finger

Influence of hand movement on skin deformation: A therapeutic glove design perspective

A study on human hand motion assessment and evaluation indices

A Contactless Method to Measure Real-Time Finger Motion Using Depth-Based Pose Estimation.

[Accuracy of Key Point Matrix Technology Based Contactless Automatic Measurement for Joint Motion of Hand].

A hand motion capture method based on infrared thermography for measuring fine motor skills in biomedicine

Evaluating the accuracy of hand models obtained from two 3D scanning techniques

Kinematic Model for Human Index Finger

Study on Movement Characteristics of Fingers During Hand Grabbing Process.

Skinmotion: What Does Skin Movement Tell Us?

2P2-Q13 Measurement of Kinesthesia Information and Contact Force at the Human Fingertip During Tracing Movement(Informative Motion & Motion Media)

Inherent Kinematics of the Human Thumb IP and MCP Joints During Functional Tasks

Experimental investigation on the centros of abduction/adduction of the metacarpophalangeal joints of human hand

Identify Finger Rotation Angles With ArUco Markers and Action Cameras

QUANTIFICATION OF THREE-DIMENSIONAL MOVEMENT OF SKIN MARKERS RELATIVE TO THE UNDERLYING BONES DURING FUNCTIONAL ACTIVITIES

2A1-P02 Measurement of the Relationship Between the Upper Limb Movement and Contact Force at Fingertip(informative Motion & Motion Media)