Abstract:Industry 5.0 emphasizes human-centered solutions. However, bridging the gap between human conditions and engineering systems remains a challenge in this era. Promoting the development of cybernetics applications in the field of rehabilitation is imperative. In the healthcare sector, the evaluation of rehabilitation products' ergonomics and the dynamic assessment of the human body are of utmost importance. Amid the burgeoning prominence of data-driven product design concepts, ergonomics methods have emerged as indispensable evaluative tools. Nevertheless, the integration of data from multiple sources remains a contemporary challenge; one must extract meaningful information from diverse data sets while enhancing data analysis and fusion techniques. In this study, we collect multimodal neurobiological signals using electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) to devise a robust multimodal rehabilitation data fusion method based on graph convolutional network (GCN), particularly focusing on the motor imagery (MI) task. To achieve a more holistic evaluation, we incorporate granger causality (GC) and brain region adjacency matrix as supplementary features, integrating electrophysiological and hemodynamic perspectives. This integration aims to enhance information complementarity and foster a comprehensive understanding of the neural processes involved in MI. Our results demonstrate the superiority of the multimodal fusion method: it achieves higher average accuracy and improved stability, as indicated by the reduced standard deviation of accuracy. This enhanced performance suggests its potential regarding broader research applications. Building upon our successful findings, we establish an intelligent algorithmic rehabilitation platform based on multimodal neural data. This platform not only holds significant clinical value, facilitating precision and personalized medicine, but also propels the advancement of medical practices, paving the way for a more tailored and effective approach to patient care. Our research makes progresses in the areas of rehabilitation modeling, equipment design, intelligence, and interdisciplinary collaboration through the in-depth study and extension of cybernetic principles and techniques.

NeuroManagement and Intelligent Computing Method on Multimodal Interaction

Current situation and prospects of neuro information systems

Artificial Intelligence in The Management of Neurodegenerative Disorders

Toward NeuroDM: Where Computational Neuroscience Meets Data Mining

A New Research Trend in Social Neuroscience: Towards an Interactive-Brain Neuroscience

A neuro-symbolic approach for multimodal reference expression comprehension

Network neuroscience for optimizing brain-computer interfaces

Brain-inspired Multimodal Learning Based on Neural Networks

Multimodal communication in animals, humans and robots: an introduction to perspectives in brain-inspired informatics

Multimodal Human–Robot Interaction for Human‐Centric Smart Manufacturing: A Survey

Multimodal English Teaching Classroom Interaction Based on Artificial Neural Network

Recent advancements in multimodal human-robot interaction

Development of a novel machine learning-based approach for brain function assessment and integrated software solution

Merging Neuroimaging and Multimedia: Methods, Opportunities, and Challenges

Brain-inspired Computing Based on Deep Learning for Human-computer Interaction: A Review

In the realm of hybrid Brain: Human Brain and AI

Multimodal Systems: Taxonomy, Methods, and Challenges

Noninvasive Human-Computer Interface Methods and Applications for Robotic Control: Past, Current, and Future

Neuroinformatics Applications of Data Science and Artificial Intelligence

Multimodal Fusion of Brain Imaging Data: Methods and Applications

Multimodal neuroimaging computing: a review of the applications in neuropsychiatric disorders