Selective attention to audiovisual speech routes activity through recurrent feedback-feedforward loops between different nodes of the speech network

Patrik Wikman,Viljami Salmela,Eetu Sjöblom,Miika Leminen,Matti Laine,Kimmo Alho

DOI: https://doi.org/10.1101/2023.07.17.549287

2024-02-12

Abstract:Selective attention related top-down modulation plays a significant role in separating relevant speech from irrelevant background speech when vocal attributes separating concurrent speakers are small and continuously evolving. Electrophysiological studies have shown that such top-down modulation enhances neural tracking of attended speech. Yet, the specific cortical regions involved remain unclear due to the limited spatial resolution of most electrophysiological techniques. To overcome such limitations, we collected both EEG (high temporal resolution) and fMRI (high spatial resolution), while human participants selectively attended to speakers in audiovisual scenes containing overlapping cocktail party speech. To utilize the advantages of the respective techniques, we analysed neural tracking of speech using the EEG data and performed representational dissimilarity-based EEG-fMRI fusion. We observed that attention enhanced neural tracking and modulated EEG correlates throughout the latencies studied. Further, attention related enhancement of neural tracking fluctuated in predictable temporal profiles. We discuss how such temporal dynamics could arise from a combination of interactions between attention and prediction as well as plastic properties of the auditory cortex. EEG-fMRI fusion revealed attention related iterative feedforward-feedback loops between hierarchically organised nodes of the ventral auditory object related processing stream. Our findings support models where attention facilitates dynamic neural changes in the auditory cortex, ultimately aiding discrimination of relevant sounds from irrelevant ones while conserving neural resources.

Neuroscience

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper attempts to explore how humans use selective attention to separate relevant speech signals and suppress the influence of irrelevant background speech in complex audio environments. Specifically, the study focuses on the following aspects:

1. **The Impact of Selective Attention on Neural Activity**: Researchers use electrophysiological techniques (such as EEG) and functional magnetic resonance imaging (fMRI) combined methods to investigate how selective attention enhances neural tracking of relevant speech and reveal the specific manifestations of this enhancement in different brain regions.

2. **Temporal Dynamic Changes**: The study finds that selective attention not only enhances neural activity but also shows nonlinear fluctuations over time. The authors discuss that these temporal dynamic changes may be caused by the interaction between attention and prediction mechanisms, as well as the plasticity characteristics of the auditory cortex.

3. **Feedback-Feedforward Loop**: Through EEG-fMRI fusion technology, the research reveals that selective attention promotes recurrent feedback-feedforward loops in the auditory object processing stream. This finding supports the model that selective attention helps dynamically alter neural activity in the auditory cortex, thereby distinguishing relevant sounds from irrelevant ones and conserving neural resources.

In summary, this paper aims to deeply understand the neural mechanisms and temporal dynamic characteristics of selective attention in complex audio environments through multimodal brain imaging techniques.