Abstract:When we comprehend language from speech, the phase of the neural response aligns with particular features of the speech input, resulting in a phenomenon referred to as neural tracking. In recent years, a large body of work has demonstrated the tracking of the acoustic envelope and abstract linguistic units at the phoneme and word levels, and beyond. However, the degree to which speech tracking is driven by acoustic edges of the signal, or by internally-generated linguistic units, or by the interplay of both, remains contentious. In this study, we used naturalistic story-listening to investigate (1) whether phoneme-level features are tracked over and above acoustic edges, (2) whether word entropy, which can reflect sentence- and discourse-level constraints, impacted the encoding of acoustic and phoneme-level features, and (3) whether the tracking of acoustic edges was enhanced or suppressed during comprehension of a first language (Dutch) compared to a statistically familiar but uncomprehended language (French). We first show that encoding models with phoneme-level linguistic features, in addition to acoustic features, uncovered an increased neural tracking response; this signal was further amplified in a comprehended language, putatively reflecting the transformation of acoustic features into internally generated phoneme-level representations. Phonemes were tracked more strongly in a comprehended language, suggesting that language comprehension functions as a neural filter over acoustic edges of the speech signal as it transforms sensory signals into abstract linguistic units. We then show that word entropy enhances neural tracking of both acoustic and phonemic features when sentence- and discourse-context are less constraining. When language was not comprehended, acoustic features, but not phonemic ones, were more strongly modulated, but in contrast, when a native language is comprehended, phoneme features are more strongly modulated. Taken together, our findings highlight the flexible modulation of acoustic, and phonemic features by sentence and discourse-level constraint in language comprehension, and document the neural transformation from speech perception to language comprehension, consistent with an account of language processing as a neural filter from sensory to abstract representations.

Neural tracking of phrases in spoken language comprehension is automatic and task-dependent

Cortical Tracking of Constituent Structure in Language Acquisition

The Influence of Linguistic Information on Cortical Tracking of Words

Delta-band Neural Activity Primarily Tracks Sentences Instead of Semantic Properties of Words

Low-frequency Cortical Activity Reflects Context-dependent Parsing of Word Sequences

Delta-band Cortical Tracking of Acoustic and Linguistic Features in Natural Spoken Narratives

Time-domain Analysis of Neural Tracking of Hierarchical Linguistic Structures

Eye Activity Tracks Task-Relevant Structures During Speech and Auditory Sequence Perception

Neural Harmonics Reflect Grammaticality

The Cortical Maps of Hierarchical Linguistic Structures During Speech Perception

Structural and sequential regularities modulate phrase-rate neural tracking

Cortical tracking of language structures: Modality-dependent and independent responses

Understanding Requires Tracking: Noise and Knowledge Interact in Bilingual Comprehension

A tradeoff between acoustic and linguistic feature encoding in spoken language comprehension

Characterizing Neural Entrainment to Hierarchical Linguistic Units using Electroencephalography (EEG).

Neural Tracking of Speech Acoustics in Noise Is Coupled with Lexical Predictability as Estimated by Large Language Models

Segmenting and Predicting Musical Phrase Structure Exploits Neural Gain Modulation and Phase Precession

Attention is Required for Knowledge-Based Sequential Grouping: Insights from the Integration of Syllables into Words.

Neural Signatures of Hierarchical Linguistic Structures in Second Language Listening Comprehension

Cortical tracking of hierarchical linguistic structures in connected speech