Abstract:OBJECTIVE: It has been shown that attentional selection in a simple dichotic listening paradigm can be decoded offline by reconstructing the stimulus envelope from single-trial neural response data. Here, we test the efficacy of this approach in an environment with non-stationary talkers. We then look beyond the envelope reconstructions themselves and consider whether incorporating the decoder values-which reflect the weightings applied to the multichannel EEG data at different time lags and scalp locations when reconstructing the stimulus envelope-can improve decoding performance.APPROACH: High-density EEG was recorded as subjects attended to one of two talkers. The two speech streams were filtered using HRTFs, and the talkers were alternated between the left and right locations at varying intervals to simulate a dynamic environment. We trained spatio-temporal decoders mapping from EEG data to the attended and unattended stimulus envelopes. We then decoded auditory attention by (1) using the attended decoder to reconstruct the envelope and (2) exploiting the fact that decoder weightings themselves contain signatures of attention, resulting in consistent patterns across subjects that can be classified.MAIN RESULTS: The previously established decoding approach was found to be effective even with non-stationary talkers. Signatures of attentional selection and attended direction were found in the spatio-temporal structure of the decoders and were consistent across subjects. The inclusion of decoder weights into the decoding algorithm resulted in significantly improved decoding accuracies (from 61.07% to 65.31% for 4 s windows). An attempt was made to include alpha power lateralization as another feature to improve decoding, although this was unsuccessful at the single-trial level.SIGNIFICANCE: This work suggests that the spatial-temporal decoder weights can be utilised to improve decoding. More generally, looking beyond envelope reconstruction and incorporating other signatures of attention is an avenue that should be explored to improve selective auditory attention decoding.

What are we really decoding? Unveiling biases in EEG-based decoding of the spatial focus of auditory attention

Decoding auditory attention (in real time) with eeg

Fast EEG-Based Decoding Of The Directional Focus Of Auditory Attention Using Common Spatial Patterns

Riemannian geometry-based decoding of the directional focus of auditory attention using EEG

Linear stimulus reconstruction works on the KU Leuven audiovisual, gaze-controlled auditory attention decoding dataset

Using Ear-EEG to Decode Auditory Attention in Multiple-speaker Environment

'Are you even listening?' - EEG-based decoding of absolute auditory attention to natural speech

EEG decoding of the target speaker in a cocktail party scenario: considerations regarding dynamic switching of talker location

A DenseNet-based method for decoding auditory spatial attention with EEG

STAnet: A Spatiotemporal Attention Network for Decoding Auditory Spatial Attention from EEG

Enhancing spatial auditory attention decoding with neuroscience-inspired prototype training

Comparison of linear and nonlinear methods for decoding selective attention to speech from ear-EEG recordings

EEG-based Auditory Attention Decoding: Towards Neuro-Steered Hearing Devices

EEG-based auditory attention decoding with audiovisual speech for hearing-impaired listeners

Auditory Attention Decoding in Four-Talker Environment with EEG

Decoding Visual Spatial Attention Control

Electroencephalogram-based Multi-class Decoding of Attended Speakers' Direction with Audio Spatial Spectrum

Improved Decoding of Attentional Selection in Multi-Talker Environments with Self-Supervised Learned Speech Representation

Neural decoding of attentional selection in multi-speaker environments without access to clean sources

Decoding of selective attention to continuous speech from the human auditory brainstem response