Abstract:Industry 5.0 emphasizes human-centered solutions. However, bridging the gap between human conditions and engineering systems remains a challenge in this era. Promoting the development of cybernetics applications in the field of rehabilitation is imperative. In the healthcare sector, the evaluation of rehabilitation products' ergonomics and the dynamic assessment of the human body are of utmost importance. Amid the burgeoning prominence of data-driven product design concepts, ergonomics methods have emerged as indispensable evaluative tools. Nevertheless, the integration of data from multiple sources remains a contemporary challenge; one must extract meaningful information from diverse data sets while enhancing data analysis and fusion techniques. In this study, we collect multimodal neurobiological signals using electroencephalogram (EEG) and functional near-infrared spectroscopy (fNIRS) to devise a robust multimodal rehabilitation data fusion method based on graph convolutional network (GCN), particularly focusing on the motor imagery (MI) task. To achieve a more holistic evaluation, we incorporate granger causality (GC) and brain region adjacency matrix as supplementary features, integrating electrophysiological and hemodynamic perspectives. This integration aims to enhance information complementarity and foster a comprehensive understanding of the neural processes involved in MI. Our results demonstrate the superiority of the multimodal fusion method: it achieves higher average accuracy and improved stability, as indicated by the reduced standard deviation of accuracy. This enhanced performance suggests its potential regarding broader research applications. Building upon our successful findings, we establish an intelligent algorithmic rehabilitation platform based on multimodal neural data. This platform not only holds significant clinical value, facilitating precision and personalized medicine, but also propels the advancement of medical practices, paving the way for a more tailored and effective approach to patient care. Our research makes progresses in the areas of rehabilitation modeling, equipment design, intelligence, and interdisciplinary collaboration through the in-depth study and extension of cybernetic principles and techniques.

A multimodal approach for fostering knowledge transfer in engineering design activity: an eye-tracking and fNIRS study

Cognitive Factors of the Transfer of Empirical Engineering Knowledge: A Behavioral and Fnirs Study.

Functional Brain Network Analysis of Knowledge Transfer While Engineering Problem-Solving.

Fostering the transfer of empirical engineering knowledge under technological paradigm shift: An experimental study in conceptual design

Transfer Of Empirical Engineering Knowledge Under Technological Paradigm Shift Evidence From An Eye-Tracking Experiment

Effects of Explaining a Science Lesson to Others or to Oneself: A Cognitive Neuroscience Approach

Crowd Intelligence Driven Design Framework Based on Perception-Retrieval Cognitive Mechanism

Research on knowledge navigation supporting rapid design of complex product

Exploring cognitive load through neuropsychological features: an analysis using fNIRS-eye tracking

The Role of Functional Near-Infrared Spectroscopy in Collaborative Research: A systematic Review

Instructor–learner neural synchronization during elaborated feedback predicts learning transfer.

Development of a novel machine learning-based approach for brain function assessment and integrated software solution

Theorizing neuro-induced relationships between cognitive diversity, motivation, grit and academic performance in multidisciplinary engineering education context

Cross-Modal Transfer Learning From EEG to Functional Near-Infrared Spectroscopy for Classification Task in Brain-Computer Interface System

Elaborated feedabck and transfer

The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation

Audio-visual cross-modality knowledge transfer for machine learning-based in-situ monitoring in laser additive manufacturing

Development of a Wearable Fnirs Sensor System to Assess Brain Activities on Cognitive Tasks

Enhancing fNIRS Analysis Using EEG Rhythmic Signatures: An EEG-Informed fNIRS Analysis Study

Frontoparietal Connectivity Neurofeedback Training for Promotion of Working Memory: An fNIRS Study in Healthy Male Participants

Close Spatial Distance and Direct Gaze Bring Better Communication Outcomes and More Intertwined Neural Networks