Abstract:Mismatch responses to tri‐modal roving stimulus sequences revealed a fronto‐parietal network of expectation violation shared across audition, somatosensation and vision. The intraparietal network hub showed specific sensitivity for cross‐modal predictability of stimulus transitions embedded in the multi‐modal sequence configuration. The brain is subjected to multi‐modal sensory information in an environment governed by statistical dependencies. Mismatch responses (MMRs), classically recorded with EEG, have provided valuable insights into the brain's processing of regularities and the generation of corresponding sensory predictions. Only few studies allow for comparisons of MMRs across multiple modalities in a simultaneous sensory stream and their corresponding cross‐modal context sensitivity remains unknown. Here, we used a tri‐modal version of the roving stimulus paradigm in fMRI to elicit MMRs in the auditory, somatosensory and visual modality. Participants (N = 29) were simultaneously presented with sequences of low and high intensity stimuli in each of the three senses while actively observing the tri‐modal input stream and occasionally reporting the intensity of the previous stimulus in a prompted modality. The sequences were based on a probabilistic model, defining transition probabilities such that, for each modality, stimuli were more likely to repeat (p = .825) than change (p = .175) and stimulus intensities were equiprobable (p = .5). Moreover, each transition was conditional on the configuration of the other two modalities comprising global (cross‐modal) predictive properties of the sequences. We identified a shared mismatch network of modality general inferior frontal and temporo‐parietal areas as well as sensory areas, where the connectivity (psychophysiological interaction) between these regions was modulated during mismatch processing. Further, we found deviant responses within the network to be modulated by local stimulus repetition, which suggests highly comparable processing of expectation violation across modalities. Moreover, hierarchically higher regions of the mismatch network in the temporo‐parietal area around the intraparietal sulcus were identified to signal cross‐modal expectation violation. With the consistency of MMRs across audition, somatosensation and vision, our study provides insights into a shared cortical network of uni‐ and multi‐modal expectation violation in response to sequence regularities.

Computational modelling of hippocampal sensitivity to expectation violation

Violation of expectation: neural correlates reflect bases of prediction

A multimodal cortical network of sensory expectation violation revealed by fMRI

A Computational Theory of Hippocampal Function, and Empirical Tests of the Theory.

Computational Models of Hippocampal Functions

A computational theory of hippocampal function, and tests of the theory: New developments

Tracking the flow of hippocampal computation: Pattern separation, pattern completion, and attractor dynamics

Pattern Separation, Completion, and Categorisation in the Hippocampus and Neocortex

The mechanisms for pattern completion and pattern separation in the hippocampus

Neocortical differentiation and hippocampal integration of non-meaningful items and their spatial location

Dissociable dynamic effects of expectation during statistical learning

Modeling of context-dependent retrieval in hippocampal region CA1: implications for cognitive function in schizophrenia

Learned response dynamics reflect stimulus timing and encode temporal expectation violations in superficial layers of mouse V1

Expectation violations produce error signals in mouse V1

Center-surround inhibition by expectation: a neuro-computational account

Expectation violations enhance neuronal encoding of sensory information in mouse primary visual cortex

Predictive Coding of Reward in the Hippocampus

Memory storage fidelity in the hippocampal circuit: the role of subregions and input statistics

A Model Of The Operation Of The Hippocampus And Entorhinal Cortex In Memory

Building and Breaking the Chain: A Model of Reward Prediction Error Integration and Segmentation of Memory

Coincidence detection of place and temporal context in a network model of spiking hippocampal neurons