Abstract:Human listeners can reliably recognize speech in complex listening environments. The underlying neural mechanisms, however, remain unclear and cannot yet be emulated by any artificial system. In this dissertation, we study how speech is represented in the human auditory cortex and how the neural representation contributes to reliable speech recognition. Cortical activity from normal hearing human subjects is noninvasively recorded using magnetoencephalography, during natural speech listening. It is first demonstrated that neural activity from auditory cortex is precisely synchronized to the slow temporal modulations of speech, when the speech signal is presented in a quiet listening environment. How this neural representation is affected by acoustic interference is then investigated. Acoustic interference degrades speech perception via two mechanisms, informational masking and energetic masking, which are addressed respectively by using a competing speech stream and a stationary noise as the interfering sound. When two speech streams are presented simultaneously, cortical activity is predominantly synchronized to the speech stream the listener attends to, even if the unattended, competing speech stream is 8 dB more intense. When speech is presented together with spectrally matched stationary noise, cortical activity remains precisely synchronized to the temporal modulations of speech until the noise is 9 dB more intense. Critically, the accuracy of neural synchronization to speech predicts how well individual listeners can understand speech in noise. Further analysis reveals that two neural sources contribute to speech synchronized cortical activity, one with a shorter response latency of about 50 ms and the other with a longer response latency of about 100 ms. The longer-latency component, but not the shorter-latency component, shows selectivity to the attended speech and invariance to background noise, indicating a transition from encoding the acoustic scene to encoding the behaviorally important auditory object, in auditory cortex. Taken together, we have demonstrated that during natural speech comprehension, neural activity in the human auditory cortex is precisely synchronized to the slow temporal modulations of speech. This neural synchronization is robust to acoustic interference, whether speech or noise, and therefore provides a strong candidate for the neural basis of acoustic background invariant speech recognition.

Auditory cortical processing in real-world listening: the auditory system going real.

Temporal coding of speech in human auditory cortex

Neural coding of continuous speech in auditory cortex during monaural and dichotic listening.

Recent advances in understanding the auditory cortex

VisibleSound: Perceiving Environmental Sound with 4D Form

Sensory and Perceptual Decisional Processes Underlying the Perception of Reverberant Auditory Environments

The Human Auditory System and Audio

Virtually spatialized sounds enhance auditory processing in healthy participants and patients with a disorder of consciousness

From Beeps to Streets: Unveiling Sensory Input and Relevance Across Auditory Contexts

A Novel Approach to Investigate Subcortical and Cortical Sensitivity to Temporal Structure Simultaneously

Treatise on Hearing: The Temporal Auditory Imaging Theory Inspired by Optics and Communication

How Does Visual Context of Urban Open Spaces Drive Soundscape Pleasantness and Eventfulness: a Laboratory Experiment Using a Higher Order Ambisonics Speaker System and Immersive Virtual Reality

Naturalistic Audiovisual Stimulation Reveals the Topographic Organization of Human Auditory Cortex

One hundred ways to process time, frequency, rate and scale in the central auditory system: a pattern-recognition meta-analysis

Neural coding strategies in auditory cortex.

Auditory information loss in real-world listening environments

Responses of auditory-cortex neurons to structural features of natural sounds

What and where in the auditory systems of sighted and early blind individuals: Evidence from representational similarity analysis

Spectral and Temporal Integration in the Auditory Cortex of Alert Primates

Neuronal mechanisms, response dynamics and perceptual functions of multisensory interactions in auditory cortex