Abstract:Studies of perception have long shown that the brain adds information to its sensory analysis of the physical environment. A touchstone example for humans is language use: to comprehend a physical signal like speech, the brain must add linguistic knowledge, including syntax. Yet, syntactic rules and representations are atemporal (i.e., abstract and not bound by time), so they must be translated into time-varying signals for speech comprehension and production. Here, we test three different models of the temporal spell-out of syntactic structure against brain activity of people listening to Dutch stories: an integratory bottom-up parser, a predictive top-down parser, and a mildly predictive left-corner parser. These models build exactly the same structure but differ in when syntactic information is added by the brain – this difference is captured in the (temporal distribution of the) complexity metric ‘incremental node count’. Using temporal response function models with both acoustic and information-theoretic control predictors, node counts were regressed against source-reconstructed delta-band activity acquired with magnetoencephalography. Neural dynamics in left frontal and temporal regions most strongly reflect node counts derived by the top-down method, which postulates syntax early in time, suggesting that predictive structure building is an important component of Dutch sentence comprehension. The absence of strong effects of the left-corner model further suggests that its mildly predictive strategy does not represent Dutch language comprehension well, in contrast to what has been found for English. Understanding when the brain projects its knowledge of syntax onto speech, and whether this is done in language-specific ways, will inform and constrain the development of mechanistic models of syntactic-structure building in the brain.

Studying language in context using the temporal generalization method

Low-frequency Cortical Activity Reflects Context-dependent Parsing of Word Sequences

A bounded accumulation model of temporal generalization outperforms existing models and captures modality differences and learning effects

Complex text processing by the temporal context machines

Time-domain Analysis of Neural Tracking of Hierarchical Linguistic Structures

Stimulating the Brain's Language Network: Syntactic Ambiguity Resolution after TMS to the Inferior Frontal Gyrus and Middle Temporal Gyrus

Exploring temporal sensitivity in the brain using multi-timescale language models: an EEG decoding study

Mind the Gap: Assessing Temporal Generalization in Neural Language Models

Brain responses to a lab-evolved artificial language with space-time metaphors

Temporal Context As Cortical Spatial Codes

The Temporal Structure of Language Processing in the Human Brain Corresponds to The Layered Hierarchy of Deep Language Models

Investigating the Timescales of Language Processing with EEG and Language Models

Neural populations in the language network differ in the size of their temporal receptive windows

Modeling structure-building in the brain with CCG parsing and large language models

Deep language algorithms predict semantic comprehension from brain activity

Neural dynamics express syntax in the time domain during natural story listening

Are Large Language Models Temporally Grounded?

Time Matters: Examine Temporal Effects on Biomedical Language Models

Information-Restricted Neural Language Models Reveal Different Brain Regions' Sensitivity to Semantics, Syntax and Context

On the Generalization Capability of Temporal Graph Learning Algorithms: Theoretical Insights and a Simpler Method