Abstract:Global aging leads to a surge in neurological diseases. Quantitative gait analysis for the early detection of neurological diseases can effectively reduce the impact of the diseases. Recently, extensive research has focused on gait-abnormality-recognition algorithms using a single type of portable sensor. However, these studies are limited by the sensor's type and the task specificity, constraining the widespread application of quantitative gait recognition. In this study, we propose a multimodal gait-abnormality-recognition framework based on a Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) network. The as-established framework effectively addresses the challenges arising from smooth data interference and lengthy time series by employing an adaptive sliding window technique. Then, we convert the time series into time–frequency plots to capture the characteristic variations in different abnormality gaits and achieve a unified representation of the multiple data types. This makes our signal processing method adaptable to several types of sensors. Additionally, we use a pre-trained Deep Convolutional Neural Network (DCNN) for feature extraction, and the consequently established CNN-BiLSTM network can achieve high-accuracy recognition by fusing and classifying the multi-sensor input data. To validate the proposed method, we conducted diversified experiments to recognize the gait abnormalities caused by different neuropathic diseases, such as amyotrophic lateral sclerosis (ALS), Parkinson's disease (PD), and Huntington's disease (HD). In the PDgait dataset, the framework achieved an accuracy of 98.89% in the classification of Parkinson's disease severity, surpassing DCLSTM's 96.71%. Moreover, the recognition accuracy of ALS, PD, and HD on the PDgait dataset was 100%, 96.97%, and 95.43% respectively, surpassing the majority of previously reported methods. These experimental results strongly demonstrate the potential of the proposed multimodal framework for gait abnormality identification. Due to the advantages of the framework, such as its suitability for different types of sensors and fewer training parameters, it is more suitable for gait monitoring in daily life and the customization of medical rehabilitation schedules, which will help more patients alleviate the harm caused by their diseases.

Hybrid Gait Recognition Method Based on Phase Trajectory

Hybrid Silhouette-skeleton Body Representation for Gait Recognition

Gait Phase Recognition of Lower Limb Based on GA Optimized BP Neural Network

Human Gait Recognition Based on Frame-by-Frame Gait Energy Images and Convolutional Long Short-Term Memory

Gait Recognition Using Multichannel Convolution Neural Networks

Human Gait Recognition Based on Self-Adaptive Hidden Markov Model

Gait Phase Detection Based on a Foot-Mounted Inertial Sensor Using Long Short-Term Memory Enhanced by Hidden Markov Model

A Novel Gait Phase Recognition Method Based on DPF-LSTM-CNN Using Wearable Inertial Sensors

Gait Phase Classification of Lower Limb Exoskeleton Based on a Compound Network Model

Wearable Device-Based Gait Recognition Using Angle Embedded Gait Dynamic Images and a Convolutional Neural Network

Gait Phase Detection Based on LSTM-CRF for Stair Ambulation

Joint Metric Learning and Hierarchical Network for Gait Recognition

HorGait: A Hybrid Model for Accurate Gait Recognition in LiDAR Point Cloud Planar Projections

Accelerometer-Based Gait Recognition by Sparse Representation of Signature Points with Clusters

Dual phase learning for large scale video gait recognition

Human Gait Recognition by Integrating Motion Feature and Shape Feature

IMU-Based Real-Time Estimation of Gait Phase Using Multi-Resolution Neural Networks

Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion

Abnormal Gait Recognition Algorithm Based on LSTM-CNN Fusion Network

Gait recognition via deep learning of the center-of-pressure trajectory

Continuous Human Gait Phase Estimation Considering Individual Diversity