Abstract:Global aging leads to a surge in neurological diseases. Quantitative gait analysis for the early detection of neurological diseases can effectively reduce the impact of the diseases. Recently, extensive research has focused on gait-abnormality-recognition algorithms using a single type of portable sensor. However, these studies are limited by the sensor's type and the task specificity, constraining the widespread application of quantitative gait recognition. In this study, we propose a multimodal gait-abnormality-recognition framework based on a Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) network. The as-established framework effectively addresses the challenges arising from smooth data interference and lengthy time series by employing an adaptive sliding window technique. Then, we convert the time series into time–frequency plots to capture the characteristic variations in different abnormality gaits and achieve a unified representation of the multiple data types. This makes our signal processing method adaptable to several types of sensors. Additionally, we use a pre-trained Deep Convolutional Neural Network (DCNN) for feature extraction, and the consequently established CNN-BiLSTM network can achieve high-accuracy recognition by fusing and classifying the multi-sensor input data. To validate the proposed method, we conducted diversified experiments to recognize the gait abnormalities caused by different neuropathic diseases, such as amyotrophic lateral sclerosis (ALS), Parkinson's disease (PD), and Huntington's disease (HD). In the PDgait dataset, the framework achieved an accuracy of 98.89% in the classification of Parkinson's disease severity, surpassing DCLSTM's 96.71%. Moreover, the recognition accuracy of ALS, PD, and HD on the PDgait dataset was 100%, 96.97%, and 95.43% respectively, surpassing the majority of previously reported methods. These experimental results strongly demonstrate the potential of the proposed multimodal framework for gait abnormality identification. Due to the advantages of the framework, such as its suitability for different types of sensors and fewer training parameters, it is more suitable for gait monitoring in daily life and the customization of medical rehabilitation schedules, which will help more patients alleviate the harm caused by their diseases.

A Multimode Markerless Gait Motion Analysis System Based on Lightweight Pose Estimation Networks

Development of a Novel Motion Capture and Gait Analysis System for Rat Locomotion

Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network–Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion

Markerless Gait Analysis Based On A Single Rgb Camera

Markerless Motion Capture System for Stroke Gait Analysis

Multimodal Gait Recognition for Neurodegenerative Diseases

PMotion: An advanced markerless pose estimation approach based on novel deep learning framework used to reveal neurobehavior

Body Sensor Network-Based Robust Gait Analysis: Toward Clinical and at Home Use

Single Camera Based Gait Analysis Method with Scaled and Image Coordinate Key-points

Automated Gait Analysis Based on a Marker-Free Pose Estimation Model

Marker-free Human Gait Analysis using a Smart Edge Sensor System

Mems Inertial Sensors Based Gait Analysis For Rehabilitation Assessment Via Multi-Sensor Fusion

Monocular 3D Human Pose Markerless Systems for Gait Assessment

Towards Wearable-Inertial-Sensor-Based Gait Posture Evaluation For Subjects With Unbalanced Gaits

Neural Networks for Pathological Gait Classification Using Wearable Motion Sensors

Markerless gait analysis through a single camera and computer vision

Video-Based Pose Estimation for Gait Analysis in Stroke Survivors during Clinical Assessments: A Proof-of-Concept Study

Regional Time-Series Coding Network and Multi-View Image Generation Network for Short-Time Gait Recognition

Depth-aware pose estimation using deep learning for exoskeleton gait analysis

Assessment of Parkinson's Motor Severity by Multi-feature and Multi-scale Motion Convolutional Neural Network with Positional Encoding

Wearable multisource quantitative gait analysis of Parkinson's diseases