Abstract:Everyday challenges impact our ability to hear and comprehend spoken language with ease, such as accented speech (source factors), spectral degradation (transmission factors), complex or unfamiliar language use (message factors), and predictability (context factors). Auditory degradation and linguistic complexity in the brain and behavior have been well investigated, and several computational models have emerged. The work here provides a novel test of the hypotheses that listening effort is partially reliant on higher cognitive auditory attention and working memory mechanisms in the frontal lobe, and partially reliant on hierarchical linguistic computation in the brain's left hemisphere. We specifically hypothesize that these models are robust and can be applied in ecologically relevant and coarse-grain contexts that rigorously control for acoustic and linguistic listening challenges. Using functional near-infrared spectroscopy during an auditory plausibility judgment task, we show the hierarchical cortical organization for listening effort in the frontal and left temporal-parietal brain regions. In response to increasing levels of cognitive demand, we found (i) poorer comprehension, (ii) slower reaction times, (iii) increasing levels of perceived mental effort, (iv) increasing levels of brain activity in the prefrontal cortex, (v) hierarchical modulation of core language processing regions that reflect increasingly higher-order auditory-linguistic processing, and (vi) a correlation between participants' mental effort ratings and their performance on the task. Our results demonstrate that listening effort is partly reliant on higher cognitive auditory attention and working memory mechanisms in the frontal lobe and partly reliant on hierarchical linguistic computation in the brain's left hemisphere. Further, listening effort is driven by a voluntary, motivation-based attention system for which our results validate the use of a single-item post-task questionnaire for measuring perceived levels of mental effort and predicting listening performance. We anticipate our study to be a starting point for more sophisticated models of listening effort and even cognitive neuroplasticity in hearing aid and cochlear implant users.

Assessing mental demand in consecutive interpreting: Insights from an fNIRS study

Predicting Fluency with Language Proficiency, Working Memory, and Directionality in Simultaneous Interpreting.

Testing the efforts model of simultaneous interpreting: An ERP study

Taking a Deeper Look at the Brain: Predicting Visual Perceptual and Working Memory Load From High-Density fNIRS Data

How Does Consecutive Interpreting Training Influence Working Memory: A Longitudinal Study of Potential Links Between the Two.

Effects of cognitive load and years of experience on phase-amplitude coupling in simultaneous interpretation

The cortical organization of listening effort: New insight from functional near-infrared spectroscopy

Analysis of selective attention processing on experienced simultaneous interpreters using EEG phase synchronization

Assessing the brain 'on the line': An ecologically-valid assessment of the impact of repetitive assembly line work on hemodynamic response and fine motor control using fNIRS.

Optical Mapping of Brain Activity Underlying Directionality and Its Modulation by Expertise in Mandarin/English Interpreting

Emotional regulation in interpreters revealed by frontal asymmetry of alpha activity

Decoding Mental Effort in a Quasi-Realistic Scenario: A Feasibility Study on Multimodal Data Fusion and Classification

ERP Neural Correlates of the Interpreter Advantage in Coordination for Interpreting Students

Investigating mental workload across task modalities: a multimodal analysis using pupillometry

Decoding working-memory load during n-Back task performance from high channel fNIRS data

Functional near-infrared spectroscopy in the evaluation of urban rail transit drivers' mental workload under simulated driving conditions

Investigating the presumed cognitive advantage of aspiring interpreters

Previous Mental Load and Incentives Influence Anticipatory Arousal as Indexed by the Baseline Pupil Diameter in a Speech-in-Noise Task

Decoding different working memory states during an operation span task from prefrontal fNIRS signals

Exploring cognitive load through neuropsychological features: an analysis using fNIRS-eye tracking

Disentangling listening effort and memory load beyond behavioural evidence: Pupillary response to listening effort during a concurrent memory task