Abstract:Adaptive behavior rests on forming predictions based on previous statistical regularities encountered in the environment. Such regularities pertain not only to the contents of the stimuli ("what") but also their timing ("when"), and both interactively modulate sensory processing. In speech streams, predictions can be formed at multiple hierarchical levels, both in terms of contents (e.g. single syllables vs. words) and timing (e.g., faster vs. slower time scales). Whether and how these hierarchies map onto each other in terms of integrating "what" and "when" predictions remains unknown. Under one hypothesis neural hierarchies may link "what" and "when" predictions within sensory processing areas: with lower cortical regions mediating interactions for smaller units e.g., syllables, and higher cortical areas mediating interactions for larger units e.g., words. Alternatively, interactions between "what" and "when" predictions might rest on a generic, sensory-independent mechanism, mediated by common attention-related (e.g., frontoparietal) networks. To address those questions, we manipulated "what" and "when" predictions at two levels - single syllables and disyllabic pseudowords - while recording neural activity using magnetoencephalography (MEG) in healthy volunteers (N=22). We studied how syllable and/or word deviants are modulated by "when" predictability, both analyzing event-related fields and using source reconstruction and dynamic causal modeling to explain the observed effects in terms of the underlying effective connectivity. "When" predictions modulated "what" mismatch responses in a specific way with regards to speech hierarchy, such that mismatch responses to deviant words (vs. syllables) were amplified by temporal predictions at a slower (vs. faster) time scale. However, these modulations were source-localized to a shared network of cortical regions, including frontal and parietal sources. Effective connectivity analysis showed that, while mismatch responses to violations of "what" predictions modulated connectivity between regions, the integration of "what" and "when" predictions selectively modulated connectivity within regions, consistent with gain effects. These results suggest that the brain integrates "what" and "when" predictions that are congruent with respect to their hierarchical level, but this integration is mediated by a shared and distributed cortical network. This contrasts with recent studies indicating separable networks for different levels of hierarchical speech processing.

Cortical-subcortical interactions underlie processing of auditory predictions measured with 7T fMRI

Temporal expectations modulate coupling between frontal and sensory brain areas

Frequency and frequency modulation share the same predictive encoding mechanisms in human auditory cortex

Systemic neurophysiological signals of auditory predictive coding

Concurrent generative models inform prediction error in the human auditory pathway

Cortical Dynamics of Acoustic and Phonological Processing in Speech Perception

Constructing the hierarchy of predictive auditory sequences in the marmoset brain

Auditory prediction errors in sound frequency and duration generated different cortical activation patterns in the human brain: an ECoG study

Predictable and unpredictable deviance detection in the human hippocampus and amygdala

Hierarchical Deviant Processing in Auditory Cortex of Awake Mice

Altered hierarchical auditory predictive processing after lesions to the orbitofrontal cortex

Intracranial Recordings and Computational Modeling of Music Reveal the Time Course of Prediction Error Signaling in Frontal and Temporal Cortices

Predictive coding in the auditory brainstem

Corticofugal Modulation of Initial Sound Processing in the Brain.

Auditory prediction and prediction error responses evoked through a novel cascade roving paradigm: a human ECoG study

Parallel mechanisms signal a hierarchy of sequence structure violations in the auditory cortex

A hierarchy of processing complexity and timescales for natural sounds in human auditory cortex

Subcortical origin of nonlinear sound encoding in auditory cortex

Oscillatory multi-timescale mechanisms underlying multisensory sequence prediction

"What" and "when" predictions jointly modulate speech processing