Abstract:<p>Available researches have confirmed that the target speech stream of human listener's selective attention in a multi-speaker scenario can be decoded from ongoing electroencephalography (EEG) signals. Due to its great potential application for the neuro-steered hearing devices, auditory attention decoding has recently received increasing attention. In practice, development of high-accuracy decoding performance and use of EEG signals with as short duration as possible are urgently needed for auditory attention decoding in real-time scenarios. However, these are still great challenges for the existing decoding methods to achieve such performances. In this paper, we propose a nonlinear modeling approach between EEG signals and speech envelopes by exploiting long short-term memory networks (LSTM), a special structure of recurrent neural networks (RNN), to decipher selective auditory attention from single-trial EEG signals, which is capable of learning long-term dependencies between EEG responses and continuous auditory stimuli. Unlike the existing linear modeling approaches, the LSTM RNN-based auditory attention decoding method is an end-to-end nonlinear decoding framework that subsumes the explicit similarity determination as part of feature learning in deep neural networks. The proposed decoding method is evaluated using the real EEG data collected from 21 subjects. Experimental results show that our proposed decoding method can achieve the state-of-the-art performance to identify the target speech in a competing two-talker scenario. When used the EEG signals with 1 s duration, the average accuracy for the 21 subjects' attention decoding is reached to 96.12%. The results show great potentials for the real-time auditory attention detection in neuro-steered hearing aids.</p>

Cross-Attention-Guided Wavenet for Mel Spectrogram Reconstruction in the ICASSP 2024 Auditory EEG Challenge

Cross-Attention-Guided WaveNet for EEG-to-MEL Spectrogram Reconstruction

ConvConcatNet: a deep convolutional neural network to reconstruct mel spectrogram from the EEG

HappyQuokka System for ICASSP 2023 Auditory EEG Challenge

Sea-Wave: Speech envelope reconstruction from auditory EEG with an adapted WaveNet

A Mel Spectrogram Enhancement Paradigm Based on CWT in Speech Synthesis

AMNet: Introducing an Adaptive Mel-Spectrogram End-to-End Neural Network for Heart Sound Classification

Auditory attention decoding from EEG-based Mandarin speech envelope reconstruction

Auditory attention decoding from electroencephalography based on long short-term memory networks

X-CrossNet: A complex spectral mapping approach to target speaker extraction with cross attention speaker embedding fusion

Self-supervised speech representation and contextual text embedding for match-mismatch classification with EEG recording

A Dual-Branch Spatio-Temporal-Spectral Transformer Feature Fusion Network for EEG-Based Visual Recognition

Enhancing spatial auditory attention decoding with neuroscience-inspired prototype training

EEG2Mel: Reconstructing Sound from Brain Responses to Music

WaveNet with Cross-Attention for Audiovisual Speech Recognition

MSFNet: Multi-Scale Fusion Network for Brain-Controlled Speaker Extraction

Detecting gamma-band responses to the speech envelope for the ICASSP 2024 Auditory EEG Decoding Signal Processing Grand Challenge

Eeg2vec: Self-Supervised Electroencephalographic Representation Learning

EEG-based Auditory Attention Detection with Spiking Graph Convolutional Network

Decoding Selective Auditory Attention with EEG Using a Transformer Model

Attention-Based Parallel Multiscale Convolutional Neural Network for Visual Evoked Potentials EEG Classification.