Abstract:Communication is challenging for disabled individuals, but with advancement of brain-computer interface (BCI) systems, alternative communication systems can be developed. Current BCI spellers, such as P300, SSVEP, and MI, have drawbacks like reliance on external stimuli or conversation irrelevant mental tasks. In contrast to these systems, Imagined speech based BCI systems rely on directly decoding the vowels/words user is thinking, making them more intuitive, user friendly and highly popular among Brain-Computer-Interface (BCI) researchers. However, more research needs to be conducted on how subject-specific characteristics such as mental state, age, handedness, nativeness and resting state activity affects the brain's output during imagined speech. In an overt speech, it is evident that native and non-native speakers' brains function differently. Therefore, this paper explores how nativeness to language affects EEG signals while imagining vowel phonemes, using brain-map analysis and scalogram and also investigates the inclusion of features extracted from resting state EEG with imagined state EEG. The Fourteen-channel EEG for Imagined Speech (FEIS) dataset was used to analyse the EEG signals recorded while imagining vowel phonemes for 16 subjects (nine native English and seven non-native Chinese). For the classification of vowel phonemes, different connectivity measures such as covariance, coherence, and Phase Synchronous Index-PSI were extracted and analysed using statistics based Multivariate Analysis of Variance (MANOVA) approach. Different fusion strategies (difference, concatenation, Common Spatial Pattern-CSP and Canonical Correlation Analysis-CCA) were carried out to incorporate resting state EEG connectivity measures with imagined state connectivity measures for enhancing the accuracy of imagined vowel phoneme recognition. Simulation results revealed that concatenating imagined state and rest state covariance and PSI features provided the maximum accuracy of 92.78% for native speakers and 94.07% for non-native speakers.

Comparative Study of Linear and Nonlinear Features Used in Imagined Vowels Classification Using a Backpropagation Neural Network Classifier

Recognition of EEG Signals from Imagined Vowels Using Deep Learning Methods

Classification of Imagined Speech Using Siamese Neural Network

Resting state EEG assisted imagined vowel phonemes recognition by native and non-native speakers using brain connectivity measures

Identification of vowels in consonant–vowel–consonant words from speech imagery based EEG signals

Imagined Speech Classification Using EEG and Deep Learning

Neural Speech Decoding During Audition, Imagination and Production

Towards Imagined Speech: Identification of Brain States from EEG Signals for BCI-based Communication Systems

Classification of imagined speech of vowels from EEG signals using multi-headed CNNs feature fusion network

Decoding Imagined Speech using Wavelet Features and Deep Neural Networks

Comparison of the prediction accuracy of machine learning algorithms in crosslinguistic vowel classification

Effect of functional and effective brain connectivity in identifying vowels from articulation imagery procedures

Vowel speech recognition from rat electroencephalography using long short-term memory neural network.

Towards Unified Neural Decoding of Perceived, Spoken and Imagined Speech from EEG Signals

Imagined speech classification exploiting EEG power spectrum features

DAL: Feature Learning from Overt Speech to Decode Imagined Speech-based EEG Signals with Convolutional Autoencoder

Decoding Imagined Speech and Computer Control using Brain Waves

A Novel Deep Learning Architecture for Decoding Imagined Speech from EEG

Objective speech intelligibility prediction using a deep learning model with continuous speech-evoked cortical auditory responses

Decoding of the speech envelope from EEG using the VLAAI deep neural network

Towards Neural Decoding of Imagined Speech based on Spoken Speech