The Interplay Between Multisensory Processing and Attention in Working Memory: Behavioral and Neural Indices of Audio-Visual Object Storage

Ceren Arslan,Daniel Schneider,Stephan Getzmann,Edmund Wascher,Laura-Isabelle Klatt
DOI: https://doi.org/10.1101/2024.03.26.586805
2024-03-29
Abstract:Although real-life events are multisensory, how audio-visual objects are stored in working memory is an open question. At a perceptual level, evidence shows that both top-down and bottom-up attentional processes can play a role in multisensory interactions. To understand how attention and multisensory processes interact in working memory, we designed an audio-visual delayed match-to-sample task in which participants were presented with one or two audio-visual memory items, followed by an audio-visual probe. In three different blocks, participants were instructed to either (a) attend to the auditory features, (b) attend to the visual features, or (c) attend to both auditory and visual features. Participants were instructed to indicate whether the task-relevant features of the probe matched one of the task-relevant feature(s) or objects in working memory. Behavioral results showed interference from task-irrelevant features, suggesting bottom-up integration of audio-visual features and their automatic encoding into working memory, irrespective of their task relevance. Yet, ERP analyses revealed no evidence for active maintenance of these task-irrelevant features, while they clearly taxed greater attentional resources during recall. Notably, alpha oscillatory activity revealed that linking information between auditory and visual modalities required more attentional demands at retrieval. Overall, these results offer critical insights into how and at which processing stage multisensory interactions occur in working memory.
Neuroscience
What problem does this paper attempt to address?