Abstract:Human listeners can reliably recognize speech in complex listening environments. The underlying neural mechanisms, however, remain unclear and cannot yet be emulated by any artificial system. In this dissertation, we study how speech is represented in the human auditory cortex and how the neural representation contributes to reliable speech recognition. Cortical activity from normal hearing human subjects is noninvasively recorded using magnetoencephalography, during natural speech listening. It is first demonstrated that neural activity from auditory cortex is precisely synchronized to the slow temporal modulations of speech, when the speech signal is presented in a quiet listening environment. How this neural representation is affected by acoustic interference is then investigated. Acoustic interference degrades speech perception via two mechanisms, informational masking and energetic masking, which are addressed respectively by using a competing speech stream and a stationary noise as the interfering sound. When two speech streams are presented simultaneously, cortical activity is predominantly synchronized to the speech stream the listener attends to, even if the unattended, competing speech stream is 8 dB more intense. When speech is presented together with spectrally matched stationary noise, cortical activity remains precisely synchronized to the temporal modulations of speech until the noise is 9 dB more intense. Critically, the accuracy of neural synchronization to speech predicts how well individual listeners can understand speech in noise. Further analysis reveals that two neural sources contribute to speech synchronized cortical activity, one with a shorter response latency of about 50 ms and the other with a longer response latency of about 100 ms. The longer-latency component, but not the shorter-latency component, shows selectivity to the attended speech and invariance to background noise, indicating a transition from encoding the acoustic scene to encoding the behaviorally important auditory object, in auditory cortex. Taken together, we have demonstrated that during natural speech comprehension, neural activity in the human auditory cortex is precisely synchronized to the slow temporal modulations of speech. This neural synchronization is robust to acoustic interference, whether speech or noise, and therefore provides a strong candidate for the neural basis of acoustic background invariant speech recognition.

On the relationship between speech and hearing

Temporal coding of speech in human auditory cortex

The relationship of speech perception and speech production: It's complicated

Dimensions Underlying the Perceived Similarity of Acoustic Environments

On the relation of speech to language

Neural Speech Decoding During Audition, Imagination and Production

Uncovering the role of semantic and acoustic cues in normal and dichotic listening

Predicting speech-in-noise ability with static and dynamic auditory figure-ground analysis using structural equation modelling

On the Relationship between Face Movements, Tongue Movements, and Speech Acoustics

Exploring the Role of Brain Oscillations in Speech Perception in Noise: Intelligibility of Isochronously Retimed Speech

Combining signal-detection theory and interaural cross-correlation approaches to account for the binaural abilities of normal-hearing listeners and listeners with “slight” hearing loss

Is Having Hearing Loss Fundamentally Different? Multigroup Structural Equation Modeling of the Effect of Cognitive Functioning on Speech Identification

Beyond acoustics -- capacity limitations of linguistic levels

The Human Auditory System and Audio

Interference of mid-level sound statistics underlie human speech recognition sensitivity in natural noise

Individual differences in the attentional modulation of the human auditory brainstem response to speech inform on speech-in-noise deficits

The Optimal Speech-to-Background Ratio for Balancing Speech Recognition With Environmental Sound Recognition

The Lateralization of Speech-Brain Coupling Is Differentially Modulated by Intrinsic Auditory and Top-Down Mechanisms

Speech perception and production

On the interplay between speech perception and production: insights from research and theories

The Structural Correlates of Statistical Information Processing during Speech Perception