Abstract:Human listeners can reliably recognize speech in complex listening environments. The underlying neural mechanisms, however, remain unclear and cannot yet be emulated by any artificial system. In this dissertation, we study how speech is represented in the human auditory cortex and how the neural representation contributes to reliable speech recognition. Cortical activity from normal hearing human subjects is noninvasively recorded using magnetoencephalography, during natural speech listening. It is first demonstrated that neural activity from auditory cortex is precisely synchronized to the slow temporal modulations of speech, when the speech signal is presented in a quiet listening environment. How this neural representation is affected by acoustic interference is then investigated. Acoustic interference degrades speech perception via two mechanisms, informational masking and energetic masking, which are addressed respectively by using a competing speech stream and a stationary noise as the interfering sound. When two speech streams are presented simultaneously, cortical activity is predominantly synchronized to the speech stream the listener attends to, even if the unattended, competing speech stream is 8 dB more intense. When speech is presented together with spectrally matched stationary noise, cortical activity remains precisely synchronized to the temporal modulations of speech until the noise is 9 dB more intense. Critically, the accuracy of neural synchronization to speech predicts how well individual listeners can understand speech in noise. Further analysis reveals that two neural sources contribute to speech synchronized cortical activity, one with a shorter response latency of about 50 ms and the other with a longer response latency of about 100 ms. The longer-latency component, but not the shorter-latency component, shows selectivity to the attended speech and invariance to background noise, indicating a transition from encoding the acoustic scene to encoding the behaviorally important auditory object, in auditory cortex. Taken together, we have demonstrated that during natural speech comprehension, neural activity in the human auditory cortex is precisely synchronized to the slow temporal modulations of speech. This neural synchronization is robust to acoustic interference, whether speech or noise, and therefore provides a strong candidate for the neural basis of acoustic background invariant speech recognition.

Speaker-Listener Neural Coupling Reveals an Adaptive Mechanism for Speech Comprehension in a Noisy Environment

Speaker-listener neural coupling reveals a right-lateralized mechanism for non-native speech-in-noise comprehension

Robust cortical encoding of slow temporal modulations of speech.

Adaptive Temporal Encoding Leads to a Background-Insensitive Cortical Representation of Speech

Understanding Requires Tracking: Noise and Knowledge Interact in Bilingual Comprehension

Temporal coding of speech in human auditory cortex

Cortical Neural Coding of Speech in Simple and Complex Auditory Scenes.

Speaker-listener neural coupling correlates with semantic and acoustic features of naturalistic speech

EEG-based Speaker-Listener Neural Coupling Reflects Speech-Selective Attentional Mechanisms Beyond the Speech Stimulus

The Compensatory Mechanism of the Prefrontal Cortex in Preserving Speech-in-Noise Comprehension among Older Adults

Leading and Following: Noise Differently Affects Semantic and Acoustic Processing During Naturalistic Speech Comprehension

Reduced Listener-speaker Neural Coupling Underlies Speech Understanding Difficulty in Older Adults

Compensatory Mechanisms for Preserving Speech-in-Noise Comprehension Involve Prefrontal Cortex in Older Adults

Auditory-Articulatory Neural Alignment Between Listener and Speaker During Verbal Communication.

Multi-talker speech comprehension at different temporal scales in listeners with normal and impaired hearing

Individual Differences in Cognition and Perception Predict Neural Processing of Speech in Noise for Audiometrically Normal Listeners.

The "two-brain" approach reveals the active role of task-deactivated default mode network in speech comprehension

Auditory and language contributions to neural encoding of speech features in noisy environments

How Does the Human Brain Process Noisy Speech in Real Life? Insights from the Second-Person Neuroscience Perspective.

Neural Mechanisms for Selectively Tuning in to the Target Speaker in a Naturalistic Noisy Situation

Neural Correlates of Early Sound Encoding and their Relationship to Speech-in-Noise Perception