A Monitoring Method of Freezing of Gait Based on Multimodal Fusion

Bochen Li,Yan Li,Yining Sun,Xianjun Yang,Xu Zhou,Zhiming Yao

DOI: https://doi.org/10.1016/j.bspc.2023.104589

IF: 5.1

2023-01-01

Biomedical Signal Processing and Control

Abstract:Freezing of Gait (FOG) is an episodic lower extremity movement disorder that is highly susceptible to falls and carries a serious risk of disability. Monitoring of FOG can assist in the diagnosis and treatment of FOG. Providing appropriate gait guidance along with monitoring can help reduce the frequency and duration of freezing epi-sodes. This study aims to improve the robustness of the monitoring model using multimodal fusion methods. The gait signals from 32 FOG patients are collected by the inertial measurement unit (IMU) and force-sensitive insole (FSI) simultaneously. A multimodal fused FOG monitoring model was constructed by using deep neural networks to extract complementary features from IMU and FSI signals respectively, and feature-level fusion of the two modalities by an adaptive weighting method. Experimental results show that the proposed multimodal fusion approach improves the F1 value by 0.029 in the FOG detection task compared to the unimodal model. In addition, to construct the pre-FOG dataset more accurately, an automatic labeling method of pre-FOG events based on the FOG index ratio is also proposed in this paper. Compared to directly labeling the data 2.5 s before the freezing episode as the pre-FOG event, the proposed labeling method obtained more samples and improved the freezing prediction accuracy by 1.4 %.