Abstract:Humans are able to identify and track a target speaker amid a cacophony of acoustic interference, an ability which is often referred to as the cocktail party phenomenon. Results from several decades of studying this phenomenon have culminated in recent years in various promising attempts to decode the attentional state of a listener in a competing-speaker environment from non-invasive neuroimaging recordings such as magnetoencephalography (MEG) and electroencephalography (EEG). To this end, most existing approaches compute correlation-based measures by either regressing the features of each speech stream to the M/EEG channels (the decoding approach) or vice versa (the encoding approach). To produce robust results, these procedures require multiple trials for training purposes. Also, their decoding accuracy drops significantly when operating at high temporal resolutions. Thus, they are not well-suited for emerging real-time applications such as smart hearing aid devices or brain-computer interface systems, where training data might be limited and high temporal resolutions are desired. In this paper, we close this gap by developing an algorithmic pipeline for real-time decoding of the attentional state. Our proposed framework consists of three main modules: (1) Real-time and robust estimation of encoding or decoding coefficients, achieved by sparse adaptive filtering, (2) Extracting reliable markers of the attentional state, and thereby generalizing the widely-used correlation-based measures thereof, and (3) Devising a near real-time state-space estimator that translates the noisy and variable attention markers to robust and statistically interpretable estimates of the attentional state with minimal delay. Our proposed algorithms integrate various techniques including forgetting factor-based adaptive filtering, ℓ<sub>1</sub>-regularization, forward-backward splitting algorithms, fixed-lag smoothing, and Expectation Maximization. We validate the performance of our proposed framework using comprehensive simulations as well as application to experimentally acquired M/EEG data. Our results reveal that the proposed real-time algorithms perform nearly as accurately as the existing state-of-the-art offline techniques, while providing a significant degree of adaptivity, statistical robustness, and computational savings.

Fast EEG-Based Decoding Of The Directional Focus Of Auditory Attention Using Common Spatial Patterns

Decoding auditory attention (in real time) with eeg

Riemannian geometry-based decoding of the directional focus of auditory attention using EEG

EEG-based Auditory Attention Decoding: Towards Neuro-Steered Hearing Devices

What are we really decoding? Unveiling biases in EEG-based decoding of the spatial focus of auditory attention

A Neural-Inspired Architecture for EEG-Based Auditory Attention Detection

Low Latency Auditory Attention Detection with Common Spatial Pattern Analysis of EEG Signals.

Detecting the Locus of Auditory Attention Based on the Spectro-Spatial-temporal Analysis of EEG.

Using Ear-EEG to Decode Auditory Attention in Multiple-speaker Environment

Auditory Attention Detection in Real-Life Scenarios Using Common Spatial Patterns from EEG

Comparison of linear and nonlinear methods for decoding selective attention to speech from ear-EEG recordings

Electroencephalogram-based Multiclass Auditory Attention Decoding of Attended Speaker Direction

EEG-Based Fast Auditory Attention Detection in Real-Life Scenarios Using Time-Frequency Attention Mechanism.

A DenseNet-based method for decoding auditory spatial attention with EEG

Electroencephalogram-based Multi-class Decoding of Attended Speakers' Direction with Audio Spatial Spectrum

Deep learning-based auditory attention decoding in listeners with hearing impairment

Auditory attention decoding from electroencephalography based on long short-term memory networks

EEG-Based Short-Time Auditory Attention Detection Using Multi-Task Deep Learning.

Real-time control of a hearing instrument with EEG-based attention decoding

Neural decoding of attentional selection in multi-speaker environments without access to clean sources

Real-Time Tracking of Selective Auditory Attention From M/EEG: A Bayesian Filtering Approach