Abstract:Human listeners can reliably recognize speech in complex listening environments. The underlying neural mechanisms, however, remain unclear and cannot yet be emulated by any artificial system. In this dissertation, we study how speech is represented in the human auditory cortex and how the neural representation contributes to reliable speech recognition. Cortical activity from normal hearing human subjects is noninvasively recorded using magnetoencephalography, during natural speech listening. It is first demonstrated that neural activity from auditory cortex is precisely synchronized to the slow temporal modulations of speech, when the speech signal is presented in a quiet listening environment. How this neural representation is affected by acoustic interference is then investigated. Acoustic interference degrades speech perception via two mechanisms, informational masking and energetic masking, which are addressed respectively by using a competing speech stream and a stationary noise as the interfering sound. When two speech streams are presented simultaneously, cortical activity is predominantly synchronized to the speech stream the listener attends to, even if the unattended, competing speech stream is 8 dB more intense. When speech is presented together with spectrally matched stationary noise, cortical activity remains precisely synchronized to the temporal modulations of speech until the noise is 9 dB more intense. Critically, the accuracy of neural synchronization to speech predicts how well individual listeners can understand speech in noise. Further analysis reveals that two neural sources contribute to speech synchronized cortical activity, one with a shorter response latency of about 50 ms and the other with a longer response latency of about 100 ms. The longer-latency component, but not the shorter-latency component, shows selectivity to the attended speech and invariance to background noise, indicating a transition from encoding the acoustic scene to encoding the behaviorally important auditory object, in auditory cortex. Taken together, we have demonstrated that during natural speech comprehension, neural activity in the human auditory cortex is precisely synchronized to the slow temporal modulations of speech. This neural synchronization is robust to acoustic interference, whether speech or noise, and therefore provides a strong candidate for the neural basis of acoustic background invariant speech recognition.

Two Stages of Speech Envelope Tracking in Human Auditory Cortex Modulated by Speech Intelligibility

Θ-Band Cortical Tracking of the Speech Envelope Shows the Linear Phase Property

Phase Property of Envelope-Tracking EEG Response is Preserved in Patients with Disorders of Consciousness.

Cortical entrainment to continuous speech: functional roles and interpretations.

Delta-band Cortical Tracking of Acoustic and Linguistic Features in Natural Spoken Narratives

Robust Cortical Entrainment to the Speech Envelope Relies on the Spectro-Temporal Fine Structure

[The Neural Encoding of Continuous Speech - Recent Advances in EEG and MEG Studies].

Robust cortical encoding of slow temporal modulations of speech.

Differential modulation of auditory responses to attended and unattended speech in different listening conditions.

Temporal coding of speech in human auditory cortex

Congruent Audiovisual Speech Enhances Cortical Envelope Tracking During Auditory Selective Attention.

Auditory and language contributions to neural encoding of speech features in noisy environments

Neural decoding of the speech envelope: Effects of intelligibility and spectral degradation

Continuous speech with pauses inserted between words increases cortical tracking of speech envelope

On the Speech Envelope in the Cortical Tracking of Speech

Envelope reconstruction of speech and music highlights stronger tracking of speech at low frequencies

Neural tracking of the speech envelope predicts binaural unmasking

FMRI speech tracking in primary and non-primary auditory cortex while listening to noisy scenes

Neural envelope tracking predicts speech intelligibility and hearing aid benefit in children with hearing loss

Cortical Oscillations In Auditory Perception And Speech: Evidence For Two Temporal Windows In Human Auditory Cortex

Time-domain Analysis of Neural Tracking of Hierarchical Linguistic Structures