Contactless Evaluation of Rigidity in Parkinson's Disease by Machine Vision and Machine Learning

Xue Zhu,Weikun Shi,Yun Ling,Ningdi Luo,Qianyi Yin,Yichi Zhang,Aonan Zhao,Guanyu Ye,Haiyan Zhou,Jing Pan,Liche Zhou,Linghao Cao,Pei Huang,Pingchen Zhang,Zhonglue Chen,Cheng Chen,Shinuan Lin,Jin Zhao,Kang Ren,Yuyan Tan,Jun Liu
DOI: https://doi.org/10.1097/cm9.0000000000002668
IF: 6.133
2023-01-01
Chinese Medical Journal
Abstract:To the Editor: Parkinson's disease (PD) is one of the most common neurodegenerative movement disorders.[1] Its cardinal symptoms include bradykinesia, tremor, and rigidity.[2] The severity of PD-related motor symptoms is usually semiquantitatively evaluated by the Movement Disorder Society-sponsored revision of the Unified Parkinson's Disease Rating Scale (MDS-UPDRS)[3] part three (MDS-UPDRS III). A series of machine perception and machine learning technologies have been applied to assist the manual rating of motor symptoms.[4,5] Machine vision, which does not require physical contact between the examiner and the patient, has the potential for real-world application in remote assessments.[5] Bradykinesia and tremor have been investigated through machine vision previously,[5] but the possibility of rigidity assessment through machine vision has not been explored. For rigidity assessment, the examiner needs to touch the patient, move the limbs, and score the severity of symptom by feeling muscle tone abnormalities. There is an urgent need for contactless evaluation of rigidity through digitally based telemedicine. Rigidity was shown to be correlated with bradykinesia in previous studies.[5] We hypothesized that motions reflecting bradykinesia might provide information that can be used in the evaluation of rigidity. Here, we explored the possibility of contactless evaluation for rigidity assessment in PD using machine vision and machine learning technologies. The study was approved by the Ruijin Hospital Ethics Committee (No. 2015-99). Written informed consent was obtained from each participant, including consent to the use and publication of pictures and videos.149 PD patients treated at Shanghai Jiao Tong University School of Medicine Ruijin Hospital from September 2020 to May 2021 were included in our study. See Supplementary Method 1, http://links.lww.com/CM9/B514 for details of inclusion and exclusion criteria. All patients were asked to complete motions in front of the camera [Supplementary Method 2, https://links.lww.com/CM9/B514], following the guideline of MDS-UPDRS III. Videos with subpar quality were excluded. 135 subjects were included in the final subject set for analysis. Rigidity–upper extremity (UE), Rigidity–lower extremity (LE), and Rigidity–neck were rated by an experienced neurologist at the recording site. The rating was later used as a label in modeling. Based on the aforementioned hypothesis we selected motions of bradykinesia for the UE, LE, and neck rigidity model [Supplementary Method 3, https://links.lww.com/CM9/B514]. Information captured by the camera was processed conducted by different machine vision algorithms [Supplementary Method 4, http://links.lww.com/CM9/B514], and features were extracted [Supplementary Method 5, http://links.lww.com/CM9/B514]. A total of 8903 features for Rigidity–UE, 6675 features for Rigidity–LE, and 68,315 features for Rigidity–neck were extracted. Each feature encompassed four dimensions: motion, joint, signal, and attribute.Motion represented the motions above, and was marked during video recording.Joint represented the related joints where features were extracted, and was automatically marked by deep learning structural algorithms. Signal was manually marked and was a type of original raw data before calculation. There were three types of signals including distance, area, and angle. Attribute encompassed speed, amplitude, decrement, asymmetry, variability, and complexity. Speed, amplitude, and decrement were as described in the original MDS-UPDRS III scale. Asymmetry indicated the difference between two sides. Variability and complexity were marked manually based on mathematical definition of features. Variability, such as the standard deviation and coefficient of variation, was an attribute representing the degree of dispersion during a whole test. Complexity summarized aspects that cannot be easily interpreted, including sample kurtosis, entropy, crest factor, clearance factor, etc. The algorithm of Extreme Gradient Boosting (XGBOOST) was applied for feature selection and model training by using rater labels and features. Three machine rating models, Rigidity–UE, Rigidity–LE, and Rigidity–neck, were built in this process. Since few samples had a rigidity score of 4 in our study, samples that scored 4 were merged with samples that scored 3. For each model, rigidity was transformed into a four-category (0, 1, 2, and 3 & 4) classifier. Feature importance revealed by the XGBOOST model represented the contributing weights of a specific feature to the rigidity evaluation [Supplementary Table 1, https://links.lww.com/CM9/B514]. When analyzing the relationship between a specific feature and the label (rigidity score), the Spearman correlation coefficient was applied to provide detailed information, including the strength and direction of the correlation. Subjects were split into a training set and an independent evaluation set by stratified sampling based on the sum of rigidity scores. Independent validation of 23 samples was performed in our study. Three metrics were used to evaluate the model performance based on calculation between the score by the expert rater and the score by the model. The intraclass correlation coefficient (ICC) was calculated to represent the consistency. The Spearman correlation coefficient (ρ) demonstrated a correlation. ACC ± 0 was the percentage of cases with no error. It was used to represent the accuracy of the models [Supplementary Method 6, https://links.lww.com/CM9/B514]. See Supplementary Method 7, https://links.lww.com/CM9/B514 for statistical analysis, Supplementary Method 8, https://links.lww.com/CM9/B514 for data availability, and Supplementary Figure 1 for a flow chart of the entire study protocol. The sample included 1350 videos from 135 subjects. The demographic information and rigidity scores of the training set (112 patients) and independent test set (23 patients) are shown in Supplementary Table 2, https://links.lww.com/CM9/B514. There was no statistically significant discrepancy in demographic data or score distribution between the two sets. In the model of Rigidity–UE, consistency between the ratings of the model and those of expert raters achieved moderate performance (ICC = 0.66, 95% confidence interval [CI]: 0.45–0.79, P <0.01). The correlation coefficient between the ratings of the model and those of expert raters was moderate (ρ = 0.64), and the absolute accuracy was 0.65 [Supplementary Table 3, https://links.lww.com/CM9/B514]. The motion with the greatest contribution was pronation–supination movements of hands (52.3%). The second was kinetic tremor of hands (16.5%) [Figure 1A], in which the motion of pointing toward and away from the nose was utilized for feature extraction instead of the tremor itself and which encompassed information for upper extremity rigidity. Regarding the joint of feature origin, the joints designated Hand (69.1%) and HandTip (28.2%) had the largest contributions [Figure 1B]. Distance (49.7%) and angle (35.9%) contributed most to the domain of signal [Figure 1C]. Variability (35.2%) and complexity (27.4%) had the highest importance in attribute, while speed (18.4%), decrement (14.5%), and amplitude (4.5%) had less significance [Figure 1D]. In this contactless evaluation system, the movement of the distal joints of the upper extremities conveyed more information for rigidity.Figure 1: Importance of features in four dimensions: (A) motion, (B) joint, (C) signal, and (D) attribute. FT: Finger tapping; HM: Hand movements; KTOH: Kinetic tremor of hands; LA: Leg agility; LE: Lower extremity; PSOH: Pronation–supination movements of hands; PTOH: Postural tremor of hands; Rig: Rigidity; TT: Toe tapping; UE: Upper extremity.In the Rigidity–LE model, the ICC was moderate at 0.60 (95% CI: 0.39–0.76, P <0.01), the Spearman correlation coefficient was moderate at 0.58 (P <0.01), and the absolute accuracy was 0.72 [Supplementary Table 3, https://links.lww.com/CM9/B514]. Gait (55.4%) and toe tapping (41.7%) were the most prominent motions [Figure 1A]. The main contributing joint was TipToe (41.7%) [Figure 1B]. Distance (92.4%) had almost all of the importance in signals [Figure 1C]. Together, variability (34.4%) and complexity (25.1%) accounted for more than half of the contributions by attributes, consistent with the model of Rigidity–UE [Figure 1D]. One example of variability feature was "13_back_angular_velocity_var_hip2ankle_angle_feature_point_feature" from the motion of gait. This meant the variance of the angular velocity of the angle behind the knee [Supplementary Figure 2A, https://links.lww.com/CM9/B514]. Lead pipe resistance caused the joint to move slowly during walking. The angular speed was more concentrated (purple and blue, Supplementary Figure 2A, https://links.lww.com/CM9/B514); thus, the variability was smaller, which indicated more rigidity. One feature of complexity extracted from the motion of gait was "13_abs_angular_velocity_kurtisis_min_knee_y_feature_point_feature". This was the minimum kurtosis of the knee velocity in the y direction, which characterizes the tailedness of the distribution of knee-raising speed [Supplementary Figure 2B, https://links.lww.com/CM9/B514]. Supplementary Figure 2B, https://links.lww.com/CM9/B514depicted the velocity of knee raising while walking. Patients with rigidity tended to raise their knees more slowly, and the velocity was more concentrated (purple and blue), with a high peak and thin tail, as shown on the left, and thus the kurtosis was larger (6.60 and 2.17). Those with less rigidity tend to raise faster, and the velocity was more disperse (red and green) with a low peak and thick tail, as shown on the right, and the kurtosis was smaller (1.39 and 0.26). The ICC was 0.70 (95% CI: 0.40–0.86, P <0.01, moderate), the Spearman correlation coefficient was moderate at 0.67, and the absolute accuracy was 0.78 for Rigidity neck [Supplementary Table 3, https://links.lww.com/CM9/B514]. Gait (41.0%) was the most important motion of origin [Figure 1A]. HandTip (46.2%) and Hand (31.4%) had the greatest contribution in joint [Figure 1B]. Distance (80.4%) was the most important signal [Figure 1C]. Similar to the previous two models, complexity (43.6%) and variability (41.3%) had the most importance among attributes [Figure 1D]. The top 10 features with the greatest importance of each model are shown in Supplementary Table 1, https://links.lww.com/CM9/B514. Previously, rigidity was judged on slow passive movement of major joints and was defined as velocity-independent resistance to passive movement.[2] Various studies of wearable sensors have analyzed the correlation between features and rigidity scores. One study using features extracted from myotonometry for building a machine learning model had an ICC greater than 0.90.[6] Our ICC was 0.60–0.70. Previous quantitative measurements of rigidity required contact because rigidity assessment was based on passive movement. Rigidity was assessed through patients' motions by camera, eliminating the need for contact in our study. It is important to note that motions reflecting bradykinesia were used to evaluate rigidity, not bradykinesia itself. For example, the "kinetic tremor of the hands" was typically assessed using the tremor subscale. However, the motion of pointing to and away from the nose can also be useful for assessing rigidity in the upper extremities, as it reflects bradykinesia. Attributes of variability and complexity could provide more objective and comprehensive features for clinical assessments, and our present study demonstrated that machine vision had the potential to capture those elements of information and accomplish rigidity assessment. Our study had certain limitations. The samples were collected from only one movement disorder clinic. Advanced PD patients (Hohen–Yahr stage 4–5) were limited (1/135, 0.7%). The lack of samples with tremor-dominant and mixed subtypes was also a limitation. Our results had large CIs due to the small independent test set. Future multicenter studies with larger and more balanced subject pools are needed to increase the generalization and adaptability of the research results. Contactless rigidity evaluation was feasible through machine vision and machine learning. The movement distance of distal joints of the body reflected proximal joint rigidity. Variability and complexity, information specific to machine vision, were the most important factors in rigidity assessment. Future studies of a contactless rating system for PD motor symptoms have the potential to aid telemedicine. Acknowledgments The authors thank the patients for their participation in this study. Funding The present study was supported by grants from the National Natural Science Foundation of China (Nos. 82230040, 82071415 [JL], and 81971183 [YYT]) and Shanghai Jiao Tong University Trans-med Awards Research, (No. 20220103 [JL]). Conflicts of interest None.
What problem does this paper attempt to address?