Abstract:The brain's circuitry for perceiving and producing speech may show a notable level of overlap that is crucial for normal development and behavior. The extent to which sensorimotor integration plays a role in speech perception remains highly controversial, however. Methodological constraints related to experimental designs and analysis methods have so far prevented the disentanglement of neural responses to acoustic versus articulatory speech features. Using a passive listening paradigm and multivariate decoding of single-trial fMRI responses to spoken syllables, we investigated brain-based generalization of articulatory features (place and manner of articulation, and voicing) beyond their acoustic (surface) form in adult human listeners. For example, we trained a classifier to discriminate place of articulation within stop syllables (e.g., /pa/ vs /ta/) and tested whether this training generalizes to fricatives (e.g., /fa/ vs /sa/). This novel approach revealed generalization of place and manner of articulation at multiple cortical levels within the dorsal auditory pathway, including auditory, sensorimotor, motor, and somatosensory regions, suggesting the representation of sensorimotor information. Additionally, generalization of voicing included the right anterior superior temporal sulcus associated with the perception of human voices as well as somatosensory regions bilaterally. Our findings highlight the close connection between brain systems for speech perception and production, and in particular, indicate the availability of articulatory codes during passive speech perception.SIGNIFICANCE STATEMENT: Sensorimotor integration is central to verbal communication and provides a link between auditory signals of speech perception and motor programs of speech production. It remains highly controversial, however, to what extent the brain's speech perception system actively uses articulatory (motor), in addition to acoustic/phonetic, representations. In this study, we examine the role of articulatory representations during passive listening using carefully controlled stimuli (spoken syllables) in combination with multivariate fMRI decoding. Our approach enabled us to disentangle brain responses to acoustic and articulatory speech properties. In particular, it revealed articulatory-specific brain responses of speech at multiple cortical levels, including auditory, sensorimotor, and motor regions, suggesting the representation of sensorimotor information during passive speech perception.

Evaluation and integration of visual and auditory information in speech perception

Human Audio-Visual Consonant Recognition Analyzed With Three Bimodal Integration Models

AUDIOVISUAL SPEECH PERCEPTION IN DIOTIC AND DICHOTIC LISTENING CONDITIONS

Prediction and constraint in audiovisual speech perception

Reassessing the Benefits of Audiovisual Integration to Speech Perception and Intelligibility

Auditory-visual integration during nonconscious perception

Neural Mechanisms Underlying Cross-Modal Phonetic Encoding

Resolving competing predictions in speech: How qualitatively different cues and cue reliability contribute to phoneme identification

Multisensory integration of speech signals: the relationship between space and time

EXPRESS: Prior multisensory learning can facilitate auditory-only voice-identity and speech recognition in noise

Multisensory benefits for speech recognition in noisy environments

On the Relationship between Face Movements, Tongue Movements, and Speech Acoustics

Decoding Articulatory Features from fMRI Responses in Dorsal Speech Regions

Distinct Mechanism of Audiovisual Integration with Informative and Uninformative Sound in a Visual Detection Task: A DCM Study

Quantified Acoustic–optical Speech Signal Incongruity Identifies Cortical Sites of Audiovisual Speech Processing

Behavioral Response Modeling to Resolve Listener- and Stimulus-Related Influences on Audiovisual Speech Integration in Cochlear Implant Users

The Dynamics of Audiovisual Behavior in Speech

Fusion of visual and acoustic signals for command-word recognition

Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study

Fusion of dichotic consonants in normal-hearing and hearing-impaired listeners

Multisensory Integration of Native and Nonnative Speech in Bilingual and Monolingual Adults