Abstract:Industry 5.0 emphasizes human-centered solutions. However, bridging the gap between human conditions and engineering systems remains a challenge in this era. Promoting the development of cybernetics applications in the field of rehabilitation is imperative. In the healthcare sector, the evaluation of rehabilitation products' ergonomics and the dynamic assessment of the human body are of utmost importance. Amid the burgeoning prominence of data-driven product design concepts, ergonomics methods have emerged as indispensable evaluative tools. Nevertheless, the integration of data from multiple sources remains a contemporary challenge; one must extract meaningful information from diverse data sets while enhancing data analysis and fusion techniques. In this study, we collect multimodal neurobiological signals using electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) to devise a robust multimodal rehabilitation data fusion method based on graph convolutional network (GCN), particularly focusing on the motor imagery (MI) task. To achieve a more holistic evaluation, we incorporate granger causality (GC) and brain region adjacency matrix as supplementary features, integrating electrophysiological and hemodynamic perspectives. This integration aims to enhance information complementarity and foster a comprehensive understanding of the neural processes involved in MI. Our results demonstrate the superiority of the multimodal fusion method: it achieves higher average accuracy and improved stability, as indicated by the reduced standard deviation of accuracy. This enhanced performance suggests its potential regarding broader research applications. Building upon our successful findings, we establish an intelligent algorithmic rehabilitation platform based on multimodal neural data. This platform not only holds significant clinical value, facilitating precision and personalized medicine, but also propels the advancement of medical practices, paving the way for a more tailored and effective approach to patient care. Our research makes progresses in the areas of rehabilitation modeling, equipment design, intelligence, and interdisciplinary collaboration through the in-depth study and extension of cybernetic principles and techniques.

EGCN++: A New Fusion Strategy for Ensemble Learning in Skeleton-Based Rehabilitation Exercise Assessment

Don't Touch What Matters: Task-Aware Lipschitz Data Augmentation for Visual Reinforcement Learning.

An Expert-Knowledge-Based Graph Convolutional Network for Skeleton- Based Physical Rehabilitation Exercises Assessment

A Skeleton-Based Rehabilitation Exercise Assessment System with Rotation Invariance

Development of a novel machine learning-based approach for brain function assessment and integrated software solution

Radiographs and Texts Fusion Learning Based Deep Networks for Skeletal Bone Age Assessment

Activity Recognition in Rehabilitation Training Based on Ensemble Stochastic Configuration Networks

Symmetrical Enhanced Fusion Network for Skeleton-Based Action Recognition

Temporal Dynamics and Physical Priori Multimodal Network for Rehabilitation Physical Training Evaluation

A Deep Learning Framework for Assessing Physical Rehabilitation Exercises

A Novel Multimodal Human-Exoskeleton Interface Based on EEG and Semg Activity for Rehabilitation Training.

Rehabilitation Exercise Quality Assessment through Supervised Contrastive Learning with Hard and Soft Negatives

Research on Multimodal Fusion Recognition Method of Upper Limb Motion Patterns

Boosting lower-limb motor imagery performance through an ensemble method for gait rehabilitation

Multimodal Human-Exoskeleton Interface for Lower Limb Movement Prediction Through a Dense Co-Attention Symmetric Mechanism.

Evaluating Convolutional Neural Networks as a Method of EEG–EMG Fusion

D-STGCNT: A Dense Spatio-Temporal Graph Conv-GRU Network based on transformer for assessment of patient physical rehabilitation

Precision Rehabilitation for Patients Post-Stroke based on Electronic Health Records and Machine Learning

Skeleton Focused Human Activity Recognition in RGB Video

3D Pose Based Feedback for Physical Exercises

Automatic Assessment of Functional Movement Screening Exercises with Deep Learning Architectures