Abstract:Abstract The aim of this study was (1) to provide behavioral evidence for multimodal feature integration in an object recognition task in humans and (2) to characterize the processing stages and the neural structures where multisensory interactions take place. Event-related potentials (ERPs) were recorded from 30 scalp electrodes while subjects performed a forced-choice reaction-time categorization task: At each trial, the subjects had to indicate which of two objects was presented by pressing one of two keys. The two objects were defined by auditory features alone, visual features alone, or the combination of auditory and visual features. Subjects were more accurate and rapid at identifying multimodal than unimodal objects. Spatiotemporal analysis of ERPs and scalp current densities revealed several auditory-visual interaction components temporally, spatially, and functionally distinct before 200 msec poststimulus. The effects observed were (1) in visual areas, new neural activities (as early as 40 msec poststimulus) and modulation (amplitude decrease) of the N185 wave to unimodal visual stimulus, (2) in the auditory cortex, modulation (amplitude increase) of subcomponents of the unimodal auditory N1 wave around 90 to 110 msec, and (3) new neural activity over the right fronto-temporal area (140 to 165 msec). Furthermore, when the subjects were separated into two groups according to their dominant modality to perform the task in unimodal conditions (shortest reaction time criteria), the integration effects were found to be similar for the two groups over the nonspecific fronto-temporal areas, but they clearly differed in the sensory-specific cortices, affecting predominantly the sensory areas of the nondominant modality. Taken together, the results indicate that multisensory integration is mediated by flexible, highly adaptive physiological processes that can take place very early in the sensory processing chain and operate in both sensory-specific and nonspecific cortical structures in different ways.

Neural Correlates of Audiovisual Integration of Semantic Category Information.

Asymmetrical Cross-Modal Influence on Neural Encoding of Auditory and Visual Features in Natural Scenes.

Crossmodal Integration Enhances Neural Representation of Task-Relevant Features in Audiovisual Face Perception

Selective Audiovisual Semantic Integration Enabled by Feature-Selective Attention.

The Behavioral Presentations of Audio-Visual Integration in Rats

Locating the Sources for Cross-Modal Interactions and Decision Making During Judging the Visual-Affected Auditory Intensity Change.

Neural Mechanisms for the Effect of Prior Knowledge on Audiovisual Integration.

Cortical networks for audiovisual interactions during visual-affected auditory discrimination

Modulation of audiovisual integration in the left and right sides: effects of side and spatial coherency

Early Cortical Connective Network Relating To Audiovisual Stimulation By Partial Directed Coherence Analysis

Neural Mechanisms of Audiovisual Integration in Integrated Processing for Verbal Perception and Spatial Factors

Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study

The auditory-visual multisensory neurons and auditory-visual information integration in rat cortex

Neural Processing of Naturalistic Audiovisual Events in Space and Time

The Effect of Visual Reliability on Auditory-Visual Integration: an Event-Related Potential Study

Auditory Dominance in Processing Chinese Semantic Abnormalities in Response to Competing Audio-visual Stimuli.

Characterizing the Time-Varying Brain Networks of Audiovisual Integration across Frequency Bands

Reproducibility And Discriminability Of Brain Patterns Of Semantic Categories Enhanced By Congruent Audiovisual Stimuli

The Neural Signature of Spatial Frequency-Based Information Integration in Scene Perception

Attention Drives Visual Processing and Audiovisual Integration During Multimodal Communication

Neural Integration of Audiovisual Sensory Inputs in Macaque Amygdala and Adjacent Regions