Abstract:Global aging leads to a surge in neurological diseases. Quantitative gait analysis for the early detection of neurological diseases can effectively reduce the impact of the diseases. Recently, extensive research has focused on gait-abnormality-recognition algorithms using a single type of portable sensor. However, these studies are limited by the sensor's type and the task specificity, constraining the widespread application of quantitative gait recognition. In this study, we propose a multimodal gait-abnormality-recognition framework based on a Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) network. The as-established framework effectively addresses the challenges arising from smooth data interference and lengthy time series by employing an adaptive sliding window technique. Then, we convert the time series into time–frequency plots to capture the characteristic variations in different abnormality gaits and achieve a unified representation of the multiple data types. This makes our signal processing method adaptable to several types of sensors. Additionally, we use a pre-trained Deep Convolutional Neural Network (DCNN) for feature extraction, and the consequently established CNN-BiLSTM network can achieve high-accuracy recognition by fusing and classifying the multi-sensor input data. To validate the proposed method, we conducted diversified experiments to recognize the gait abnormalities caused by different neuropathic diseases, such as amyotrophic lateral sclerosis (ALS), Parkinson's disease (PD), and Huntington's disease (HD). In the PDgait dataset, the framework achieved an accuracy of 98.89% in the classification of Parkinson's disease severity, surpassing DCLSTM's 96.71%. Moreover, the recognition accuracy of ALS, PD, and HD on the PDgait dataset was 100%, 96.97%, and 95.43% respectively, surpassing the majority of previously reported methods. These experimental results strongly demonstrate the potential of the proposed multimodal framework for gait abnormality identification. Due to the advantages of the framework, such as its suitability for different types of sensors and fewer training parameters, it is more suitable for gait monitoring in daily life and the customization of medical rehabilitation schedules, which will help more patients alleviate the harm caused by their diseases.

Assessment of Parkinson's Motor Severity by Multi-feature and Multi-scale Motion Convolutional Neural Network with Positional Encoding

A Wearable Multi-sensor System for Classification of Multiple System Atrophy and Parkinson's Disease

Exploring Motion Boundaries in an End-to-End Network for Vision-based Parkinson's Severity Assessment

Video-Based Quantification of Gait Impairments in Parkinson's Disease Using Skeleton-Silhouette Fusion Convolution Network

Vision-based Estimation of MDS-UPDRS Gait Scores for Assessing Parkinson's Disease Motor Severity

Assessing gait dysfunction severity in Parkinson's Disease using 2-Stream Spatial-Temporal Neural Network

Global joint information extraction convolution neural network for Parkinson's disease diagnosis

Static-Dynamic Temporal Networks for Parkinson's Disease Detection and Severity Prediction

Data-driven Gait based Severity Classification for Parkinson's Disease using Duo Spatiotemporal Convoluted Kernel Boosted ResNet model

Quantifying Parkinson’s disease motor severity under uncertainty using MDS-UPDRS videos

Multi-Shared-Task Self-Supervised CNN-LSTM for Monitoring Free-Body Movement UPDRS-III Using Wearable Sensors

Cross-Spatiotemporal Graph Convolution Networks for Skeleton-Based Parkinsonian Gait MDS-UPDRS Score Estimation

Automated assessment of Parkinsonian finger-tapping tests through a vision-based fine-grained classification model

Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion

Parkinson's severity diagnosis explainable model based on 3D multi-head attention residual network

Early diagnosis and clinical score prediction of Parkinson's disease based on longitudinal neuroimaging data

Integrated Equipment for Parkinson’s Disease Early Detection Using Graph Convolution Network

Automatic Parkinson's Disease Diagnosis with Wearable Sensor Technology for Medical Robot

Deep 1D-Convnet for accurate Parkinson disease detection and severity prediction from gait

Multimodal Gait Recognition for Neurodegenerative Diseases

An Automatic Evaluation Method for Parkinson's Dyskinesia Using Finger Tapping Video for Small Samples