Abstract:Brain-computer interface (BCI) technologies have been widely used in many areas. In particular, non-invasive technologies such as electroencephalography (EEG) or near-infrared spectroscopy (NIRS) have been used to detect motor imagery, disease, or mental state. It has been already shown in literature that the hybrid of EEG and NIRS has better results than their respective individual signals. The fusion algorithm for EEG and NIRS sources is the key to implement them in real-life applications. In this research, we propose three fusion methods for the hybrid of the EEG and NIRS-based brain-computer interface system: linear fusion, tensor fusion, and $p$th-order polynomial fusion. Firstly, our results prove that the hybrid BCI system is more accurate, as expected. Secondly, the $p$th-order polynomial fusion has the best classification results out of the three methods, and also shows improvements compared with previous studies. For a motion imagery task and a mental arithmetic task, the best detection accuracy in previous papers were 74.20\% and 88.1\%, whereas our accuracy achieved was 77.53\% and 90.19\% . Furthermore, unlike complex artificial neural network methods, our proposed methods are not as computationally demanding.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is how to improve the performance of the brain - computer interface (BCI) system by fusing electroencephalogram (EEG) and near - infrared spectroscopy (NIRS) signals. Specifically, the paper proposes three fusion methods: Linear Fusion (LF), Tensor Fusion (TF), and p - th - order Polynomial Fusion (pth - PF) to achieve the effective combination of EEG and NIRS signals, thereby obtaining higher classification accuracy in Motor Imagery (MI) tasks and Mental Arithmetic (MA) tasks.
### Background and Motivation
- **Background**: BCI technology is widely used in many fields, such as robot control, workload detection, or brain disease detection. Non - invasive BCI technologies (such as EEG and NIRS) are favored because of their high safety and convenience. However, these non - invasive signals usually contain a large amount of noise, and how to accurately detect these signals remains a challenge.
- **Motivation**: Existing research shows that using EEG and NIRS signals in combination can achieve better results than using either signal alone. Therefore, developing effective fusion algorithms is the key to achieving this goal.
### Research Objectives
- **Improve classification accuracy**: Improve the classification accuracy of MI and MA tasks by proposing new fusion methods.
- **Reduce the demand for computing resources**: Compared with complex deep neural network methods, the proposed fusion methods have lower computing resource requirements and are suitable for real - time applications.
### Main Contributions
- **Propose three fusion methods**: Linear fusion, tensor fusion, and p - th - order polynomial fusion.
- **Experimental verification**: Experiments were carried out using the benchmark data set of Jaeyoung Shin et al. to verify the effectiveness of the proposed methods.
- **Performance improvement**: In the MI task, the best classification accuracy rate reached 77.53%, while in the MA task, the best classification accuracy rate reached 90.19%, both of which are better than existing research.
### Method Overview
- **Feature extraction**: Use a convolutional neural network (CNN) to extract feature vectors from EEG, oxy - NIRS, and deoxy - NIRS signals.
- **Fusion methods**:
- **Linear fusion**: Simply concatenate all feature vectors.
- **Tensor fusion**: Obtain a feature tensor containing the interaction of multiple feature vectors through the outer product operation.
- **p - th - order polynomial fusion**: Consider the interaction within and between feature vectors and generate richer features through polynomial fusion.
### Experimental Results
- **Classification accuracy**: In the MI task, the p - th - order polynomial fusion method has the highest classification accuracy rate, reaching 77.53%; in the MA task, the classification accuracy rate is the highest, reaching 90.19%.
- **Comparative analysis**: Compared with existing research, the proposed methods show significant advantages in classification accuracy and have lower computing resource requirements.
### Conclusion
The paper successfully improves the classification accuracy of EEG and NIRS signals in the BCI system by proposing three fusion methods. In particular, the p - th - order polynomial fusion method shows the best performance in both tasks, proving its potential in practical applications.