Abstract:Global aging leads to a surge in neurological diseases. Quantitative gait analysis for the early detection of neurological diseases can effectively reduce the impact of the diseases. Recently, extensive research has focused on gait-abnormality-recognition algorithms using a single type of portable sensor. However, these studies are limited by the sensor's type and the task specificity, constraining the widespread application of quantitative gait recognition. In this study, we propose a multimodal gait-abnormality-recognition framework based on a Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) network. The as-established framework effectively addresses the challenges arising from smooth data interference and lengthy time series by employing an adaptive sliding window technique. Then, we convert the time series into time–frequency plots to capture the characteristic variations in different abnormality gaits and achieve a unified representation of the multiple data types. This makes our signal processing method adaptable to several types of sensors. Additionally, we use a pre-trained Deep Convolutional Neural Network (DCNN) for feature extraction, and the consequently established CNN-BiLSTM network can achieve high-accuracy recognition by fusing and classifying the multi-sensor input data. To validate the proposed method, we conducted diversified experiments to recognize the gait abnormalities caused by different neuropathic diseases, such as amyotrophic lateral sclerosis (ALS), Parkinson's disease (PD), and Huntington's disease (HD). In the PDgait dataset, the framework achieved an accuracy of 98.89% in the classification of Parkinson's disease severity, surpassing DCLSTM's 96.71%. Moreover, the recognition accuracy of ALS, PD, and HD on the PDgait dataset was 100%, 96.97%, and 95.43% respectively, surpassing the majority of previously reported methods. These experimental results strongly demonstrate the potential of the proposed multimodal framework for gait abnormality identification. Due to the advantages of the framework, such as its suitability for different types of sensors and fewer training parameters, it is more suitable for gait monitoring in daily life and the customization of medical rehabilitation schedules, which will help more patients alleviate the harm caused by their diseases.

Recognition method based on multi-information fusion for gait patterns of above-knee prosthesis

Gait Phase Recognition of Lower Limb Based on GA Optimized BP Neural Network

Gait-Event-Based human intention recognition approach for lower limb

Gait Recognition Using Optical Motion Capture: A Decision Fusion Based Method

Gait Recognition Via Multiple Features and Views Information Fusion

Transferable multi-modal fusion in knee angles and gait phases for their continuous prediction

A Novel Gait Phase Recognition Method Based on DPF-LSTM-CNN Using Wearable Inertial Sensors

Design of a Bio-Inspired Gait Phase Decoder Based on Temporal Convolution Network Architecture With Contralateral Surface Electromyography Toward Hip Prosthesis Control

Feature Decoupling for Multimodal Locomotion and Estimation of Knee and Ankle Angles Implemented by Multi-Model Fusion

Research on the Application of Multi-Source Information Fusion in Multiple Gait Pattern Transition Recognition

Gait phase recognition of lower limb exoskeleton system based on the integrated network model

Gait Recognition for Lower Limb Exoskeletons Based on Interactive Information Fusion

Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion

Gait-stride-and-frequency-based human intention recognition approach and experimental verification on lower limb exoskeleton

Hybrid Gait Recognition Method Based on Phase Trajectory

A Multi-Stage Adaptive Feature Fusion Neural Network for Multimodal Gait Recognition

Design and Control of a Lower Limb Rehabilitation Robot Based on Human Motion Intention Recognition with Multi-Source Sensor Information

Motion Intent Recognition in Intelligent Lower Limb Prosthesis Using One-Dimensional Dual-Tree Complex Wavelet Transforms

Research on the sensing system of lower limb exoskeleton robot based on multi-information fusion

Research on Multimodal Fusion Recognition Method of Upper Limb Motion Patterns

Locomotion Prediction for Lower Limb Prostheses in Complex Environments Via Semg and Inertial Sensors