Data augmentation for Convolutional LSTM based brain computer interface system

Kahoko Takahashi,Zhe Sun,Jordi Solé-Casals,Andrzej Cichocki,Anh Huy Phan,Qibin Zhao,Hui-Hai Zhao,Shangkun Deng,Ruggero Micheletto

DOI: https://doi.org/10.1016/j.asoc.2022.108811

IF: 8.7

2022-04-01

Applied Soft Computing

Abstract:Electroencephalogram (EEG) is a noninvasive method to detect spatio-temporal electric signals in human brain, actively used in the recent development of Brain Computer Interfaces (BCI). EEG's patterns are affected by the task, but also other variable factors influence the subject focus on the task and result in noisy EEG signals difficult to decipher. To surpass these limitations methods based on artificial neural networks (ANNs) are used, they are inherently robust to noise and do not require models. However, they learn from examples and require lots of training data-sets. This will increase costs, need research time and subjects effort. To reduce the number of experiments necessary for network training, we devised a methodology to provide artificial data from a limited number of training data-sets. This was done by applying Empirical Mode Decomposition (EMD) on the EEG frames and intermixing their Intrinsic Mode Function (IMFs). We experimented on motor imagery (MI) tests where participants were asked to imagine movement of the left (or right) arm while under EEG recording. The EEG data were firstly transformed using the Morlet wavelet and then fed to an originally designed Convolutional Neural Network (CNN) with long short term memory blocks (LSTM-RNN). The introduction of artificial frames improved performances when compared with standard algorithms. The artificial frames become advantageous even when the number of available real frames was only of 7 or 8. In a test with two subjects (200 recordings for each subjects), we reached an accuracy better than 88% for both subjects. Improvements due to the artificial data were especially noticeable for the under-performing subject, whose EEG had lower accuracy. Imagination recognition accuracy was about 89% with 360 training frames, in which 300 were artificially created starting from 60 real ones. We believe this methodology of synthesizing artificial data may contribute to the development of novel and more efficient ways to train neural networks for brain computer interfaces.

computer science, artificial intelligence, interdisciplinary applications