Abstract:Highlights • We propose a model to detect the spatial locus of a listener's auditory attention. • EEG is modeled as a graph with node features based on time-frequency feature fusion. • Graph convolution can take advantage of local brain neuroanatomical connectivity. • Global attention mechanism is adopted to learn global brain effective connectivity. • Right-lateralized frontotemporal activation and connectivity exist in spatial hearing. Auditory spatial attention detection (ASAD) aims to decipher the spatial locus of a listener's selective auditory attention from electroencephalogram (EEG) signals. However, current models may exhibit deficiencies in EEG feature extraction, leading to overfitting on small datasets or a decline in EEG discriminability. Furthermore, they often neglect topological relationships between EEG channels and, consequently, brain connectivities. Although graph-based EEG modeling has been employed in ASAD, effectively incorporating both local and global connectivities remains a great challenge. To address these limitations, we propose a new ASAD model. First, time-frequency feature fusion provides a more precise and discriminative EEG representation. Second, EEG segments are treated as graphs, and the graph convolution and global attention mechanism are leveraged to capture local and global brain connections, respectively. A series of experiments are conducted in a leave-trials-out cross-validation manner. On the MAD-EEG and KUL datasets, the accuracies of the proposed model are more than 9% and 3% higher than those of the corresponding state-of-the-art models, respectively, while the accuracy of the proposed model on the SNHL dataset is roughly comparable to that of the state-of-the-art model. EEG time-frequency feature fusion proves to be indispensable in the proposed model. EEG electrodes over the frontal cortex are most important for ASAD tasks, followed by those over the temporal lobe. Additionally, the proposed model performs well even on small datasets. This study contributes to a deeper understanding of the neural encoding related to human hearing and attention, with potential applications in neuro-steered hearing devices.

Shared and distinct neural signatures of feature and spatial attention

Common Neural Mechanisms Control Attention and Working Memory

Overlapping neural representations for selection in attention and working memory

Spatiotemporal and sensory modality attention processing with domain-specific representations in frontoparietal areas

Focal vs Diffuse: Mechanisms of attention mediated performance enhancement in a hierarchical model of the visual system

Decoding Visual Feature Versus Visual Spatial Attention Control with Deep Neural Networks

Time-resolved Neural Effects of Attention Precuing and Spatial Location in the Left and Right Fusiform Face Areas.

Distinguishing the Neural Mechanism of Attentional Control and Working Memory in Feature-Based Attentive Tracking.

Mapping dynamic spatial patterns of brain function with spatial-wise attention

Common and distinct neural mechanisms of attention

Spatiotemporal Dynamics of Attention Networks Revealed by Representational Similarity Analysis of EEG and fMRI

Functional brain networks predicting sustained attention are not specific to perceptual modality

Selective Visual Attention Revealed by Integrating FMRI and Eye-Tracking

Connectome-based Models Predict Separable Components of Attention in Novel Individuals

Spatial and feature-selective attention interact to drive selective coding in frontoparietal cortex

Functional MRI Studies of the Neural Mechanisms of Human Brain Attentional Networks

Brain connectivity and time-frequency fusion-based auditory spatial attention detection

A Spatial–Channel–Temporal-Fused Attention for Spiking Neural Networks

Neural mechanisms of top-down divided and selective spatial attention in visual and auditory perception

Behavioral and neural mechanisms of face-specific attention during goal-directed visual search

Spatial and Feature-selective Attention Have Distinct, Interacting Effects on Population-level Tuning