Abstract:Objectives Understanding brain processing of auditory and visual speech is essential for advancing speech perception research and improving clinical interventions for individuals with hearing impairment. Functional near-infrared spectroscopy (fNIRS) is deemed to be highly suitable for measuring brain activity during language tasks. However, accurate data interpretation also requires validated stimuli and behavioral measures. Design Twenty-six adults with normal hearing listened to sentences from the Oldenburg Sentence Test (OLSA), and brain activation in the temporal, occipital, and prefrontal areas was measured by fNIRS. The sentences were presented in one of the four different modalities: speech-in-quiet, speech-in-noise, audiovisual speech or visual speech (i.e., lipreading). To support the interpretation of our fNIRS data, and to obtain a more comprehensive understanding of the study population, we performed hearing tests (pure tone and speech audiometry) and collected behavioral data using validated questionnaires, in-task comprehension questions, and listening effort ratings. Results In the auditory conditions (i.e., speech-in-quiet and speech-in-noise), we observed cortical activity in the temporal regions bilaterally. During the visual speech condition, we measured significant activation in the occipital area. Following the audiovisual condition, cortical activation was observed in both regions. Furthermore, we established a baseline for how individuals with normal hearing process visual cues during lipreading, and we found higher activity in the prefrontal cortex in noise conditions compared to quiet conditions, linked to higher listening effort. Conclusions We demonstrated the applicability of a clinically inspired audiovisual speech-comprehension task in participants with normal hearing. The measured brain activation patterns were supported and complemented by objective and behavioral parameters.

fMRI investigation of sentence comprehension by eye and by ear: Modality fingerprints on cognitive processes

Modality independence of word comprehension

Time course of fMRI-activation in language and spatial networks during sentence comprehension

Task-Dependent Modulation Of Regions In The Left Temporal Cortex During Auditory Sentence Comprehension (Vol 584, Pg 351, 2015)

Supramodal Sentence Processing in the Human Brain: fMRI Evidence for the Influence of Syntactic Complexity in More Than 200 Participants

Basic Linguistic Composition Recruits the Left Anterior Temporal Lobe and Left Angular Gyrus During Both Listening and Reading

Task-modulated Activation and Functional Connectivity of the Temporal and Frontal Areas During Speech Comprehension

Fractionating difficulty during sentence comprehension using functional neuroimaging

The anatomy of language: a review of 100 fMRI studies published in 2009

Brain correlates of discourse processing: An fMRI investigation of irony and conventional metaphor comprehension

Conversational production and comprehension: fMRI-evidence reminiscent of but deviant from the classical Broca–Wernicke model

Cortical cognitive processing during reading captured using functional-near infrared spectroscopy

Modulation of brain activity by psycholinguistic information during naturalistic speech comprehension and production

Localising semantic and syntactic processing in spoken and written language comprehension: An Activation Likelihood Estimation meta-analysis

Lipreading a naturalistic narrative in a female population: Neural characteristics shared with listening and reading

Connectivity of Fronto-Temporal Regions in Syntactic Structure Building During Speaking and Listening

Brain Activation Patterns in Normal Hearing Adults: An fNIRS Study Using an Adapted Clinical Speech Comprehension Task

Neural substrates of narrative comprehension and memory

The Cerebellum Is Sensitive to the Lexical Properties of Words During Spoken Language Comprehension

Modality-Specific and Amodal Language Processing by Single Neurons

The Sounds of Sentences: Differentiating the Influence of Physical Sound, Sound Imagery, and Linguistically Implied Sounds on Physical Sound Processing