Abstract:Mismatch responses to tri‐modal roving stimulus sequences revealed a fronto‐parietal network of expectation violation shared across audition, somatosensation and vision. The intraparietal network hub showed specific sensitivity for cross‐modal predictability of stimulus transitions embedded in the multi‐modal sequence configuration. The brain is subjected to multi‐modal sensory information in an environment governed by statistical dependencies. Mismatch responses (MMRs), classically recorded with EEG, have provided valuable insights into the brain's processing of regularities and the generation of corresponding sensory predictions. Only few studies allow for comparisons of MMRs across multiple modalities in a simultaneous sensory stream and their corresponding cross‐modal context sensitivity remains unknown. Here, we used a tri‐modal version of the roving stimulus paradigm in fMRI to elicit MMRs in the auditory, somatosensory and visual modality. Participants (N = 29) were simultaneously presented with sequences of low and high intensity stimuli in each of the three senses while actively observing the tri‐modal input stream and occasionally reporting the intensity of the previous stimulus in a prompted modality. The sequences were based on a probabilistic model, defining transition probabilities such that, for each modality, stimuli were more likely to repeat (p = .825) than change (p = .175) and stimulus intensities were equiprobable (p = .5). Moreover, each transition was conditional on the configuration of the other two modalities comprising global (cross‐modal) predictive properties of the sequences. We identified a shared mismatch network of modality general inferior frontal and temporo‐parietal areas as well as sensory areas, where the connectivity (psychophysiological interaction) between these regions was modulated during mismatch processing. Further, we found deviant responses within the network to be modulated by local stimulus repetition, which suggests highly comparable processing of expectation violation across modalities. Moreover, hierarchically higher regions of the mismatch network in the temporo‐parietal area around the intraparietal sulcus were identified to signal cross‐modal expectation violation. With the consistency of MMRs across audition, somatosensation and vision, our study provides insights into a shared cortical network of uni‐ and multi‐modal expectation violation in response to sequence regularities.

Top-down modulation of sensory processing and mismatch in the mouse posterior parietal cortex

Multimodal mismatch responses in mouse auditory cortex

Experience-dependent predictions of feedforward and contextual information in mouse visual cortex

Feedback inhibition derived from the posterior parietal cortex regulates the neural properties of the mouse visual cortex

Learning-Induced Odor Modulation of Neuronal Activity in Auditory Cortex

A multimodal cortical network of sensory expectation violation revealed by fMRI

Hierarchical Deviant Processing in Auditory Cortex of Awake Mice

Top-Down Modulation of Lateral Interactions in Visual Cortex

Spatiotemporal dynamics across visual cortical laminae support a predictive coding framework for interpreting mismatch responses

A latent pool of neurons silenced by sensory-evoked inhibition can be recruited to enhance perception

Rule-based modulation of a sensorimotor transformation across cortical areas

Auditory, Visual, and Cross-Modal Mismatch Negativities in the Rat Auditory and Visual Cortices

Posterior parietal cortex represents sensory history and mediates its effects on behaviour

Sudden sensory events trigger modality-independent responses across layers in the mouse neocortex

Posteromedial cortical networks encode visuomotor prediction errors.

Mesoscale cortical dynamics reflect the interaction of sensory evidence and temporal expectation during perceptual decision-making

Attentional modulation of secondary somatosensory and visual thalamus of mice

Posterior parietal cortex contributions to cross-modal brain plasticity upon sensory loss

Interactive Auditory Task Reveals Complex Sensory-Action Integration in Mouse Primary Auditory Cortex

Modulations of Insular Projections by Prior Belief Mediate the Precision of Prediction Error During Tactile Learning.