Abstract:At any moment in time, streams of information reach the brain through the different senses. Given this wealth of noisy information, it is essential that we select information of relevance - a function fulfilled by attention - and infer its causal structure to eventually take advantage of redundancies across the senses. Yet, the role of selective attention during causal inference in cross-modal perception is unknown. We tested experimentally whether the distribution of attention across vision and touch enhances cross-modal spatial integration (visual-tactile ventriloquism effect, Expt. 1) and recalibration (visual-tactile ventriloquism aftereffect, Expt. 2) compared to modality-specific attention, and then used causal-inference modeling to isolate the mechanisms behind the attentional modulation. In both experiments, we found stronger effects of vision on touch under distributed than under modality-specific attention. Model comparison confirmed that participants used Bayes-optimal causal inference to localize visual and tactile stimuli presented as part of a visual-tactile stimulus pair, whereas simultaneously collected unity judgments - indicating whether the visual-tactile pair was perceived as spatially-aligned - relied on a sub-optimal heuristic. The best-fitting model revealed that attention modulated sensory and cognitive components of causal inference. First, distributed attention led to an increase of sensory noise compared to selective attention toward one modality. Second, attending to both modalities strengthened the stimulus-independent expectation that the two signals belong together, the prior probability of a common source for vision and touch. Yet, only the increase in the expectation of vision and touch sharing a common source was able to explain the observed enhancement of visual-tactile integration and recalibration effects with distributed attention. In contrast, the change in sensory noise explained only a fraction of the observed enhancements, as its consequences vary with the overall level of noise and stimulus congruency. Increased sensory noise leads to enhanced integration effects for visual-tactile pairs with a large spatial discrepancy, but reduced integration effects for stimuli with a small or no cross-modal discrepancy. In sum, our study indicates a weak a priori association between visual and tactile spatial signals that can be strengthened by distributing attention across both modalities.

The Modulation of Expectation Violation on Attention: Evidence from the Spatial Cueing Effects

The Neglected Contribution of Memory Encoding in Spatial Cueing: A New Theory of Costs and Benefits.

Effects of Location-based and Object-based Selective Attention in Visual Target Detection Tasks

Encoding suppression: Linking spatial cueing costs to the attentional blink

The Cueing Effect of Exogenous Attentional Orienting on Stereopsis: an ERP Study

Superior Attentional Efficiency of Auditory Cue via the Ventral Auditory-thalamic Pathway

Precision-dependent modulation of social attention

A dissociation between the effects of expectations and attention in selective visual processing

Reward expectation modulates N2pc for target selection: Electrophysiological evidence

Causal Evidence for Expression of Perceptual Expectations in Category-Selective Extrastriate Regions

Attentional Modulation of Speed-Change Perception in the Perifoveal and Near-Peripheral Visual Field

Cueing spatial attention through timing and probability

Differential modulation of visual responses by distractor or target expectations

Differentiating Spatial and Object-Based Effects on Attention: an Event-Related Brain Potential Study-With Peripheral Cueing

Effects of Attention Direction and Perceptual Distraction Within Visual Working Memory

Guiding spatial attention by multimodal reward cues

Enhancing rhythmic temporal expectations: The dominance of auditory modality under spatial uncertainty

Modality-specific attention attenuates visual-tactile integration and recalibration effects by reducing prior expectations of a common source for vision and touch

Supramodal and modality-specific mechanisms for stimulus-driven shifts of auditory and visual attention

Computational modelling of hippocampal sensitivity to expectation violation

Center-surround inhibition by expectation: a neuro-computational account