Abstract:\textit{Objective:} Conventional EEG-based auditory attention detection (AAD) is achieved by comparing the time-varying speech stimuli and the elicited EEG signals. However, in order to obtain reliable correlation values, these methods necessitate a long decision window, resulting in a long detection latency. Humans have a remarkable ability to recognize and follow a known speaker, regardless of the spoken content. In this paper, we seek to detect the attended speaker among the pre-enrolled speakers from the elicited EEG signals. In this manner, we avoid relying on the speech stimuli for AAD at run-time. In doing so, we propose a novel EEG-based attended speaker detection (E-ASD) task. \textit{Methods:} We encode a speaker's voice with a fixed dimensional vector, known as speaker embedding, and project it to an audio-derived voice signature, which characterizes the speaker's unique voice regardless of the spoken content. We hypothesize that such a voice signature also exists in the listener's brain that can be decoded from the elicited EEG signals, referred to as EEG-derived voice signature. By comparing the audio-derived voice signature and the EEG-derived voice signature, we are able to effectively detect the attended speaker in the listening brain. \textit{Results:} Experiments show that E-ASD can effectively detect the attended speaker from the 0.5s EEG decision windows, achieving 99.78\% AAD accuracy, 99.94\% AUC, and 0.27\% EER. \textit{Conclusion:} We conclude that it is possible to derive the attended speaker's voice signature from the EEG signals so as to detect the attended speaker in a listening brain. \textit{Significance:} We present the first proof of concept for detecting the attended speaker from the elicited EEG signals in a cocktail party environment. The successful implementation of E-ASD marks a non-trivial, but crucial step towards smart hearing aids.

Speaker Identification using EEG

Speaker Recognition with Voice Evoked EEG

Understanding effect of speech perception in EEG based speech recognition systems

Multimodal Speech Recognition Using EEG and Audio Signals: A Novel Approach for Enhancing ASR Systems

Spoken Speech Enhancement using EEG

Speaker Identification from emotional and noisy speech data using learned voice segregation and Speech VGG

Efficient Automatic Speech Recognition from EEG Signals Using Optimal Deep Learning Approach

Speech Recognition using EEG signals recorded using dry electrodes

Speech Synthesis using EEG

Brain Signals to Rescue Aphasia, Apraxia and Dysarthria Speech Recognition

EEG-informed attended speaker extraction from recorded speech mixtures with application in neuro-steered hearing prostheses

Predicting Different Acoustic Features from EEG and towards direct synthesis of Audio Waveform from EEG

NeuroHeed: Neuro-Steered Speaker Extraction using EEG Signals

Speech2EEG: Leveraging Pretrained Speech Model for EEG Signal Recognition

EEG decoding with conditional identification information

Continuous Silent Speech Recognition using EEG

Robust Speaker Identification In Noise Using Missing Data Technique And Auditory Masking

A Novel framework of EEG-based user identification by analyzing music-listening behavior

EEG-Derived Voice Signature for Attended Speaker Detection

Improving EEG based continuous speech recognition using GAN

Imagined Speech Classification Using EEG and Deep Learning