Smartphone-based Detection of Early Parkinson’s Disease with Tapping Records and a Multimodal-Multiscale Ensemble Network

Tongyue He,Junxin Chen,Yongyong Chen
DOI: https://doi.org/10.1109/jsen.2024.3452092
IF: 4.3
2024-01-01
IEEE Sensors Journal
Abstract:Potential Parkinson’s disease (PD) patients have increased dramatically in the aging society, and it is necessary to explore an automated detection method for early PD patients. Smartphones, with extremely high ownership rates, instant communication, abundant sensors, and high-speed network access, provide opportunities for the daily detection of early PD. This paper develops a multimodal-multiscale ensemble network (MMSENet) to detect early PD patients using smartphone-tapping records. Specifically, the acceleration data-converted images and preprocessed coordinate data are fed into cascaded multi-head convolutional neural network (MH-CNN) blocks for feature extraction, followed by feature fusion and classification, with the final results based on both data types being integrated through a result ensemble module. The MH-CNN block consists of two convolutional heads and a residual connection to extract multi-scale features. A novel intra-modal feature fusion module based on attention mechanisms is designed to fuse features from different images. The results ensemble module considers the meta-model probability and mean probability to improve the robustness and reliability of the detection results. Experimental results demonstrate that MMSENet can accurately detect early PD patients, achieving an AUC of 0.995, an F1 of 0.895, and an accuracy of 0.965.
What problem does this paper attempt to address?