A multimodal approach to estimating vigilance in SSVEP-based BCI
Kangning Wang,Shuang Qiu,Wei Wei,Yukun Zhang,Shengpei Wang,Huiguang He,Minpeng Xu,Tzyy-Ping Jung,Dong Ming
DOI: https://doi.org/10.1016/j.eswa.2023.120177
IF: 8.5
2023-04-29
Expert Systems with Applications
Abstract:Brain-computer interface (BCI) is a communication system that allows a direct connection between the human brain and external devices, which is able to provide assistance and improve the quality of life for people with disabilities. Vigilance is an important cognitive state and plays an important role in human–computer interaction. In BCI tasks, the low-vigilance state of the BCI user would lead to the performance degradation. Therefore, it is desirable to develop an efficient method to estimate the vigilance state of BCI users. In this study, we built a 4-target BCI system based on steady-state visual evoked potential (SSVEP) for cursor control. Electroencephalogram (EEG) and electrooculogram (EOG) were recorded simultaneously from 18 subjects during a 90-min continuous cursor-control BCI task. We proposed a multimodal vigilance estimating network, named MVENet, to estimate the vigilance state of BCI users through the multimodal signals. In this architecture, a spatial-temporal convolution module with an attention mechanism was adopted to explore the temporal-spatial information of the EEG features, and a long short-term memory module was utilized to learn the temporal dependencies of EOG features. Moreover, a fusion mechanism was built to fuse the EEG representations and EOG representations effectively. Experimental results showed that the proposed network achieved a better performance than the compared methods. These results demonstrate the feasibility and effectiveness of our methods for estimating the vigilance state of BCI users.
computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science