A survey on encoding and decoding technology of non-invasive brain-computer interface
Qiu Shuang,Yang Banghua,Chen Xiaogang,Wang Yijun,Xu Minpeng,Lyu Baoliang,Gao Xiaorong,He Huiguang
DOI: https://doi.org/10.11834/jig.230031
2023-01-01
Journal of Image and Graphics
Abstract:脑—机接口（brain-computer interface，BCI）系统通过采集、分析大脑信号，将其转换为输出指令，从而跨越外周神经系统，实现由大脑信号对外部设备的直接控制，进而用于替代、修复、增强、补充或改善中枢神经系统的正常输出。非侵入式脑—机接口由于具有安全性以及便携性等优点，得到了广泛关注和持续研究。研究人员对脑信号编码方法的不断探索扩展了BCI系统的应用场景和适用范围。同时，脑信号解码方法的不断研发极大地克服了脑电信号信噪比低的缺点，提高了系统性能，这都为构建高性能脑—机接口系统奠定了基础。本文综述了非侵入式脑—机接口编解码技术以及系统应用的最新研究进展，展望其未来发展前景，以期促进BCI系统的深入研究与广泛应用。;A brain-computer interface(BCI) establishes a direct communication pathway between the living brain and an external device in terms of brain signals-acquiring and analysis and commands-converted output. It can be used to replace, repair, augment, supplement or improve the normal output of the central nervous system. The BCIs can be divided into invasive or noninvasive BCIs based on the placement of the acquisition electrode. An invasive BCI is linked to its records or brain neurons-relevant stimulation through surgical brain-implanted electrodes. But, it is just used for animal experiments or severe paralysis patients although invasive BCIs can be used to record a high signal-to-noise ratio-related brain signals. Non-invasive BCIs have its potentials of their credibility and portability in comparison with invasive BCIs. The electroencephalography(EEG) is commonly used for brain signal for BCIs now. The EEG-based BCI systems consist of two directions:coding methods used to generate brain signals and the decoding methods used to decode brain signals. In recent years, the growth of coding methods has extended the application scenarios and applicability of the system. Furthermore, to get high-performance BCIs, brain signal decoding methods has greatly developed and low signal-to-noise ratio of EEG signals is optimized as well. The BCI systems can be segmented into such categories of active, reactive or passive. For an active BCI, a user can consciously control mental activities of external stimuli excluded. The motor imagery based(MI-based) BCI system is an active BCI system in terms of EEG signals-within specific frequency changes, which can be balanced using non-motor output mental rehearsal of a motor action. For reactive BCI, brain activity is triggered by an external stimulus, and the user reacts to the stimulus from the external world. Most researches are focused on static-state visual evoked potential based(SSVEP-based) BCI and an event-related potential based(ERP-based) BCI. SSVEPs are brain responses that are elicited over the visual region when a user focuses on a flicker of a visual stimulus. An ERP-based BCI is usually based on a P300 component of ERP, which can be produced after the onset of the stimulus. For a passive BCI, it can provide the hidden state of the brain in the human-computer interaction process rather than temporal and humanized interaction, especially for affective BCIs and mental load BCIs. Our literature analysis is focused on BCI systems from four contexts of its MI-based, SSVEP-based, ERP-based, and affective BCI. Current situation and the application of BCI systems can be analyzed for coding and decoding technology as mentioned below:1) MI-based BCI studies are mostly focused on the classification of EEG patterns during MI tasks from different limbs, such as the left hand, right hand, and both feet. However, small instruction sets are still challenged for the actual application requirements. Thus, fine MI tasks from the unilateral limb are developed to deal with that. We review fine MI paradigms of multiple unilateral limb contexts in relevance to its joints, directions, and tasks. Decoding methods consist of two procedures in common:feature extraction and feature classification. Feature extraction extracts task-related and recognizable features from brain signals, and feature classification uses features-extracted to clarify the intentions of users. For MI decoding technology, two-stage traditional decoding methods are first briefly introduced, e. g., common spatial pattern(CSP) and linear discriminant analysis(LDA). After that, we summarize the latest deep learning methods, which can improve the decoding accuracy, and some migration learning methods are summarized, which can alleviate its calibration data. Finally, recent applications of the MI-BCI system are introduced in related to control of the mechanical arm and stroke rehabilitation. 2) SSVEP-based BCI systems are concerned about in terms of their high information transmission rate, strong stability, and applicability. Recent researches to highlight of SSVEP-based BCI studies are reviewed literately as well. For example, some new coding strategies are implemented for instruction set or users' preference and such emerging decoding methods are used to optimize its performance of SSVEP detection. Additionally, we summarize the main application directions of SSVEP-based BCI systems, including communication, control, and state monitoring, and the latest application progress are reviewed as well. 3) For ERP-based BCIs, the signal-to-noise ratio of the ERP response is required to be relatively higher, and a single target-related rapid serial visual presentation paradigm is challenged for the detection of multiple targets. To sum up, BCI-hybrid paradigms are developed in terms of the integration of the ERP and other related paradigms. Function-based ERP decoding methods are divided into four categories of signal de-noising, feature extraction, transfer learning, and zero calibration algorithms. Current status of decoding methods and ERP-based BCI applications are summarized as well. 4) Affective BCI can be adopted to recognize and balance human emotion. The emotion and new decoding methods are evolved in, including multimodal fusion and transfer learning. Additionally, the application of affective BCI in healthcare is reviewed and analyzed as well. Finally, future research direction of non-invasive BCI are predicted further.