A Hybrid Brain-Computer Interface Combining P300 Potentials and Emotion Patterns for Detecting Awareness in Patients with Disorders of Consciousness

Jiahui Pan,Lei Wang,Haiyun Huang,Jun Xiao,Fei Wang,Qimei Liang,Chengwei Xu,Yuanqing Li,Qiuyou Xie

DOI: https://doi.org/10.1109/tcds.2022.3213194

IF: 4.546

2022-01-01

IEEE Transactions on Cognitive and Developmental Systems

Abstract:In this study, a hybrid brain–computer interface (BCI) system combining P300 potential and emotion patterns was proposed to improve the performance of awareness detection. Two video clips were flashed randomly to evoke the P300 potential, while a laughing or crying video clip was used to induce the corresponding emotion pattern. The subjects were asked to concentrate on the laughing or crying video clip cued by the instruction and to count the flashes of the corresponding video clip. Two layers of classification were developed. In the first layer, P300 detection and emotion recognition were performed separately using two support vector machine (SVM) classifiers. Specifically, the activation, spatial, and connection patterns were fused in emotion recognition. In the second layer, the SVM scores of P300 detection and emotion recognition were fed into another SVM classifier to determine which video clip the subjects responded to. Six healthy subjects and eight patients with disorders of consciousness (DOC) were involved in the command-following experiment. The results showed that the accuracy of the hybrid BCI system was better than those of the single-modality systems. Furthermore, three patients were able to perform tasks (66%–72%) using our hybrid BCI, which indicated their residual awareness and emotion-related abilities.

robotics,computer science, artificial intelligence,neurosciences