Abstract:Human listeners can reliably recognize speech in complex listening environments. The underlying neural mechanisms, however, remain unclear and cannot yet be emulated by any artificial system. In this dissertation, we study how speech is represented in the human auditory cortex and how the neural representation contributes to reliable speech recognition. Cortical activity from normal hearing human subjects is noninvasively recorded using magnetoencephalography, during natural speech listening. It is first demonstrated that neural activity from auditory cortex is precisely synchronized to the slow temporal modulations of speech, when the speech signal is presented in a quiet listening environment. How this neural representation is affected by acoustic interference is then investigated. Acoustic interference degrades speech perception via two mechanisms, informational masking and energetic masking, which are addressed respectively by using a competing speech stream and a stationary noise as the interfering sound. When two speech streams are presented simultaneously, cortical activity is predominantly synchronized to the speech stream the listener attends to, even if the unattended, competing speech stream is 8 dB more intense. When speech is presented together with spectrally matched stationary noise, cortical activity remains precisely synchronized to the temporal modulations of speech until the noise is 9 dB more intense. Critically, the accuracy of neural synchronization to speech predicts how well individual listeners can understand speech in noise. Further analysis reveals that two neural sources contribute to speech synchronized cortical activity, one with a shorter response latency of about 50 ms and the other with a longer response latency of about 100 ms. The longer-latency component, but not the shorter-latency component, shows selectivity to the attended speech and invariance to background noise, indicating a transition from encoding the acoustic scene to encoding the behaviorally important auditory object, in auditory cortex. Taken together, we have demonstrated that during natural speech comprehension, neural activity in the human auditory cortex is precisely synchronized to the slow temporal modulations of speech. This neural synchronization is robust to acoustic interference, whether speech or noise, and therefore provides a strong candidate for the neural basis of acoustic background invariant speech recognition.

Differential Processing of Consonance and Dissonance within the Human Superior Temporal Gyrus

Spatial organization of EEG coherence during listening to consonant and dissonant chords

Early processing of consonance and dissonance in human auditory cortex

Deviance detection is the dominant component of auditory contextual processing in the lateral superior temporal gyrus: A human ECoG study

Differential fMRI Responses in the Left Posterior Superior Temporal Gyrus and Left Supramarginal Gyrus to Habituation and Change Detection in Syllables and Tones

A Double Dissociation between Anterior and Posterior Superior Temporal Gyrus for Processing Audiovisual Speech Demonstrated by Electrocorticography

Dissonance encoding in human inferior colliculus covaries with individual differences in dislike of dissonant music

Converging Evidence From Electrocorticography and BOLD fMRI for a Sharp Functional Boundary in Superior Temporal Gyrus Related to Multisensory Speech Processing

Temporal coding of speech in human auditory cortex

Neural Correlates of Consonance, Dissonance, and the Hierarchy of Musical Pitch in the Human Brainstem

Probing the neural dynamics of musicians' and non-musicians' consonant/dissonant perception: joint analyses of Electrical EncephaloGram (EEG) and Functional Magnetic Resonance Imaging (fMRI)

Controlling for individual differences in fMRI brain activation to tones, syllables, and words

Activation in the Right Inferior Parietal Lobule Reflects the Representation of Musical Structure beyond Simple Pitch Discrimination

The Causal Role of Left and Right Superior Temporal Gyri in Speech Perception in Noise: A Transcranial Magnetic Stimulation Study

Functional magnetic resonance imaging measure of automatic and controlled auditory processing

Dissonant music engages early visual processing

How Auditory Experience Differentially Influences the Function of Left and Right Superior Temporal Cortices

Cortical Oscillations In Auditory Perception And Speech: Evidence For Two Temporal Windows In Human Auditory Cortex

Functional Connections Between Auditory Cortex on Heschl's Gyrus and on the Lateral Superior Temporal Gyrus in Humans

Spontaneous Neural Activity in the Superior Temporal Gyrus Recapitulates Tuning for Speech Features

Dissociated Roles of the Inferior Frontal Gyrus and Superior Temporal Sulcus in Audiovisual Processing: Top-Down and Bottom-Up Mismatch Detection