Abstract:Industry 5.0 emphasizes human-centered solutions. However, bridging the gap between human conditions and engineering systems remains a challenge in this era. Promoting the development of cybernetics applications in the field of rehabilitation is imperative. In the healthcare sector, the evaluation of rehabilitation products' ergonomics and the dynamic assessment of the human body are of utmost importance. Amid the burgeoning prominence of data-driven product design concepts, ergonomics methods have emerged as indispensable evaluative tools. Nevertheless, the integration of data from multiple sources remains a contemporary challenge; one must extract meaningful information from diverse data sets while enhancing data analysis and fusion techniques. In this study, we collect multimodal neurobiological signals using electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) to devise a robust multimodal rehabilitation data fusion method based on graph convolutional network (GCN), particularly focusing on the motor imagery (MI) task. To achieve a more holistic evaluation, we incorporate granger causality (GC) and brain region adjacency matrix as supplementary features, integrating electrophysiological and hemodynamic perspectives. This integration aims to enhance information complementarity and foster a comprehensive understanding of the neural processes involved in MI. Our results demonstrate the superiority of the multimodal fusion method: it achieves higher average accuracy and improved stability, as indicated by the reduced standard deviation of accuracy. This enhanced performance suggests its potential regarding broader research applications. Building upon our successful findings, we establish an intelligent algorithmic rehabilitation platform based on multimodal neural data. This platform not only holds significant clinical value, facilitating precision and personalized medicine, but also propels the advancement of medical practices, paving the way for a more tailored and effective approach to patient care. Our research makes progresses in the areas of rehabilitation modeling, equipment design, intelligence, and interdisciplinary collaboration through the in-depth study and extension of cybernetic principles and techniques.

Multisensor Data Fusion And Control For Complex Neural Interface System

Improved Classification Performance of EEG-fNIRS Multimodal Brain-Computer Interface Based on Multi-Domain Features and Multi-Level Progressive Learning

Multi-modal bioelectrical signal fusion analysis based on different acquisition devices and scene settings: Overview, challenges, and novel orientation

Development of a novel machine learning-based approach for brain function assessment and integrated software solution

A novel multimodal approach for hybrid brain-computer interface

Feature Extraction and Classification Algorithm of Brain-computer Interface Based on Human Brain Central Nervous System

Multi-sensor fusion for body sensor network in medical human–robot interaction scenario

Multimodal Multiresolution Data Fusion Using Convolutional Neural Networks for IoT Wearable Sensing

A Review on Multisensor Data Fusion for Wearable Health Monitoring

Fusionatt: Deep Fusional Attention Networks For Multi-Channel Biomedical Signals

A Graph-Based Feature Extraction Algorithm Towards a Robust Data Fusion Framework for Brain-Computer Interfaces

MCFHNet: Multi-Channel Fusion Hybrid Network for Efficient EEG-fNIRS Multi-modal Motor Imagery Decoding

A Novel Multi-Sensor Hybrid Fusion Framework

Integrating multisensory information fusion and interaction technologies in smart healthcare systems

Research Based on Multi-sensor Data Fusion Technology

Simultaneous EEG-fNIRS Data Classification Through Selective Channel Representation and Spectrogram Imaging

Multimodal Fusion of EEG and Fmri for Epilepsy Detection

Decentralized Network Structure System for Multisensor Data Fusion Based on Neural Network

A Survey of the EEG/fMRI Fusion Analysis: Head Models, Methods and Applications

Dual Attentive Fusion for EEG-based Brain-Computer Interfaces

Strategic Integration: A Cross-Disciplinary Review of the fNIRS-EEG Dual-Modality Imaging System for Delivering Multimodal Neuroimaging to Applications